DOCUMENT RESUME 



ED 326 659 



CE 056 418 



AUTHOR 
TITLE 

INSTITUTION 

S?ONS AGENCY 
PUB DATE 
NOTE 



PUB TYPE 

EDRS PRICE 
DESCRIPTORS 



Fellenz, Robert A., Ed.; Conti, Gary J*, Ed* 
Intelligence and Adult Learning. 
Montana State Univ., Bozeman. Center for Adult 
Learning Research. 

Kellogg Foundation/ Battle Creek, Mich. 
Mar 90 

47p.; Papers from an Institute sponsored by the 
Center for Adult Learning Research (Bozeman, MT, 
August 1988). For related documents, see CE 056 
416-419. 

Collected Works - Conference Proceedings (021) 
MF01/PC02 Plus Postage. 

Adult Development; *Adult Education; *Adult Learning; 
Aging (Individuals); ^Cognitive Style; Intellectual 
Development; * Intelligence; ^Learning Strategies; 
^Problem Solving 



ABSTRACT 

"Understanding Adult Intelligence" (Robert Sternberg) 
focuses on the nature of intelligence. It explains Sternberg's 
triarchic theory, in which he posits three main aspects of 
intelligence: its relation to the internal or mental world of the 
learner, its relation to experience, and its relation to the 
surrounding world. "Strategies and Learning" (Claire Weinstein) 
presents a brief overview of the kinds of learning strategies that 
have been identified that differentiate between niore and less 
successful learners. A brief introduction is also given to an 
instrument used in a learning-to-le\rn course that helps to give some 
assessment of where students are in this area. "Reaction to 
Presentations: A Panel of Adult Education Professors" (Huey Long et 
al*) offers comments in an attempt to give focus to further 
d; scussicns of intelligence and learning strategies. "Real Life vs. 
Academic Problem Solving" (Robert Sternberg) discusses the difference 
between everyday problems and academic or test-taking problems. The 
differences are that academic problems are prerecognized, predefined, 
and well-structured; most school problems have one right answer; 
academic problems provide relevant information; in school settings, 
there is clear feedback; and schools emphasize individual problem 
solving. (YLB) 



****************** *************************************************** 

* Reproductions supplied by EDRS are the best that can bo wade 

* from the original document. 



Intelligence 
and 

Adult Learning 



Intelligence and Adult Learning 



Papers from an Institute Sponsored by 
The Center for Adult Learning Research 
Montana State University 
Bozeman, Montana 



Funded by 
The W. K Kellogg Foundation 



Editors 
Robert A. Fellenz 
Gary J. Conti 



March-1990 
Center For Adult Learning 
Montana State University 
Bozeman, Montana 



4 



© Kellogg Center for Adult Learning Research 
Montana State University 
All Rights Reserved 
1990 



Table of Contents 



Foreword 

Robert A. Fellenz v 

Understanding Adult Intelligence 

Robert J. Sternberg 1 

Strategies and Learning 

Claire E. Weinstein 17 

Reaction to Presentations: A Panel of Adult Education Professors 

Huey Long, Sharan Merriam, Burton Sisco, Chere Coggins 29 

Real Life vs. Academic Problem Solving 

Robert J. Sternberg 35 



iii 



Foreword 



Intelligence has been a troublesome con- 
struct for adult educators. Most research 
studies on the subject have examined the 
development or levels of intellectual ability in 
children, adolescents, or occasionally college 
students. Those few studies directed toward 
mature learners often demonstrated a basic 
bias regarding aging and ability by framing 
problems in a negative mode, such as ques- 
tioning which intellectual abilities remained 
constant through adulthood or at what age in- 
telligence began to decline. For the prac- 
titioner, applications of theory on intelligence 
to adult learning situations was frustrating. 
Prior experience, value sets, and perceived 
needs appeared to impact much more sig- 
nificantly on learning than vague motions of 
intellectual ability, Admittedly, formal educa- 
tion programs did use intelligence tests for 
admission purposes, but many teachers wit- 
nessed little correlation between such scores 
and learning in the classroom. Thus, over both 
research and practice hung the troublesome 
connotation that intelligence studies suggested 
that old dogs had trouble learning new tricks. 

But things have begun to change. During 
the past decade theorizing on intelligence has 
added a new dimension. Information process- 
ing approaches to the analysis of intellectual 
abilities have opened two avenues of study 
that are particularly relevant to those working 
with adult learners. The first is shift from 
the analysis of factors that comprise intel- 
ligence to an examination of the many ways in 
•.vhich intelligence is applied. This naturally 
has led to a study of intelligence in real-life 
situations. 

Robert Sternberg is definitely a leader in 
this effort to analyze intellectual ability in its 
relationship to adult life. In his triarchic 
theory he has posited three main aspects of 
intelligence: (a) its relation to the internal or 
mental world of the learner, (b) its relation to 
experience, and (c) its relation to the sur- 
rounding world. His relation of intelligence to 
the inner world correlate'; to the traditional 
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approach of analyzing the components in- 
volved in mental processes; but with an infor- 
mation processing stance and an insistence 
that real-life applications be taken into ac- 
count. The experiential subtheory con- 
centrates on the ability to react to situations 
with relative novelty and to automatize cer- 
tain aspects of information processing. The 
third aspect of intelligence concerns the 
ability to react to the external world or con- 
text of the individual. The three processes 
Sternberg identifies here are adaptation to, 
shaping of, and selection of environments. 

The application of intellectual functions to 
learning in the real world opens for examina- 
tion various areas relevant to adult education. 
For example, it moves the whole discussion of 
intelligence from centering on children in for- 
mal school settings to adults and the sweeping 
aspects of learning and living. Moreover, if in- 
telligence is comprised not solely of internal 
mental components acquired through 
heredity, educators may be able to effect in- 
creases in intellectual ability. Great ramifica- 
tions for continuing professional education, 
for adult basic education, and for adult 
development are inherent in this. Explana- 
tions of the decline of intellectual ability in 
aging and that disturbing concept of crystal- 
lized and fluid intelligence which are linked to 
that limited concept of intelligence as solely 
internal also are open to reexamination and 
reinterpretation. But what may be even more 
important to adult educators is that the whole 
realm of learning how to learn is elevated in 
importance. 

Claire Weinstein believes that adults can 
be taught learning strategies that will improve 
their ability to learn. She has not only 
provided leadership to the conceptualization 
of learning strategies but has also developed a 
device to measure the degree to which stu- 
dents use such strategies. Although her work 
at the University of Texas has been mainly 
with undergraduate students, she sees great 
potential for the application of learning 
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strategies by adults to their daily learning 
tasks. Her work also has application to self- 
direction in learning for it gives the inde- 
pendent learner a scheme for evaluating their 
learning processes. 

The papers included in this monograph 
were presented to a group of adult education 
professors ai a summer institute held Mon- 
tana State University in August of 1988. 
Sternberg and Weinstein both made formal 
presentations and interacted in small group 
sessions with the adult educations. Several 
days were spent discussing the application of 



these concepts to theory and practice related 
to adult learning. An additional resource 
available to institute participants and to 
readers of this monograph was the annotated 
bibliography assembled by Jeri Hosick and 
Linda Loehr, graduate students working with 
Rosemary Caffarella at Virginia Common- 
wealth University. Both the institute and this 
monograph were sponsored by the Center for 
Adult Learning Research and were supported 
by a grant from the W.K. Kellogg Foundation. 

—Robert A. Fellenz 
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Understanding Adult Intelligence 



Robert J. Sternberg 



The topic of my talk is the nature of intel- 
ligence, and I would like to explain my triar- 
chic theory. I will start with a story that I 
think illustrates the basic precepts of the 
theory. If you get the story, you will get some 
of those precepts and some practical informa- 
tion about intelligence. 

Two college students are walking in the 
forest. As they walk they encounter a 
problem. The problem is a big, ferocious look- 
ing grizzly bear coming ht them. The first of 
the guys is somebody who is considered very 
smart in terms of standard conceptions of in- 
telligence. His IQ is very high; his ACT scores 
are very high; his teachers think he is really 
smart; his grades in school are very high. He 
looks very good in terms of intelligence in al- 
most any way you can think of; in fact, he 
shows that he is very good by calculating that 
the grizzly bear will overtake them in 7.3 
seconds. He is able to manipulate the time, 
the rate, the distance and use the formula to 
figure that out. He does the calculation and 
understandably panics because he realizes it 
looks very grim. He looks over at the second 
guy who is not very smart by any of the usual 
testing standards. His ACT scores aren't very 
good; his teachers don't think he is very 
smart; his grades aren't that good. The best 
you could say about him is that he is kind of 
shrewd or he has a lot of common sense. The 
first guy notices with disgust that the second 
guy is not panicking and he doesn't even seem 
to be scared. In fact, he's taking off his hiking 
boots and putting on jogging shoes. So the 
first guy says to the second guy, "You must be 
crazy. You'll never be able to outrun that 
grizzly bear." The second guy says to the first 
guy, "That's true; but all I have to do Is outrun 
you!" The result, of course, is that the first fel- 
low gets eaten alive and the second guy jogs 
off to safety. His kind of intelligence proved 
to be somewhat adaptive for the purposes 
needed. 

I'll tell you the sequel to the story. Under- 
standably, the second fellow who lived ac- 



quired a phobia of grizzly bears. He didn't 
want to go back up to the mountains but he 
realized that wasn't practical because of where 
he lived. Being shrewd, he decided that the 
only way to handle the problem was to go 
back up into the mountains and do deep 
relaxation and self-suggestion until he over- 
came the fear of grizzly bears. So that's what 
he did; he went back to the forest and he did 
deep relaxation and hypnotised himself and 
began to feel the fear of grizzly bears starting 
to leave him. As he was relaxing, he was 
warned of another true example of adult 
education, and that was whereas lightning 
never strikes twice in the same place; grizzly 
bears often do. As he was relaxing, a grizzly 
bear came pounding at him. He realized he 
couldn't outrun the first grv because that guy 
was dead. He couldn't clu.*b a tree because, 
although with fake stories there's always a 
tree you can climb, in true ones like this, you 
can't get up there in time. Nothing seemed to 
be working. Finally he decided that he would 
do what was left to him which was to get 
down on his knees and pray. He prayed that 
the grizzly bear would become a good Chris- 
tian like himself. It worked somewhat because 
just as the grizzly was about to attack and eat 
him alive, the bear got down on its haunches 
and started to pray. It prayed, "I thank the 
Lord for the offering I'm about to receive." 
And that was the end of the second boy. 

The point, of course, to these little 
vignettes is that there seems to be more to in- 
telligence, whether it's adult intelligence or 
practical intelligence, than what you would 
measure by standard tests. The question one 
would want to deal with is, what is intelligence 
in the broad sense? 

The question of "what is intelligence" was 
originally addressed by Alfred Benet in 
France at the turn of the century. He was 
asked to devise a test that would distinguish 
kids who were mentally retarded from kids 
who were behavior problems. The problem 
was that the teachers were putting kids with 



benavioral problems in with the mentally 
retarded kids because they wanted to get the 
kids out of their classes. That *vas a fairly 
auspicious beginning, but it had a somewhat 
unpleasant consequence in that intelligence 
came to be defined as related to what was 
done in school. Academics is only part of the 
story of intelligence. 

Throughout most of this century, the 
popular definition of intelligence has been 
that intelligence is what IQ tests test. So that 
has basically been the going view of intel- 
ligence. Not that m^ny people would admit 
that; but they nevertheless acted as though it 
was true. If you were in a school system with 
a gifted program, they wouldn't say that what 
you needed to get into the program was an IQ 
of 130, but in fact that's very likely how they 
operated. What it gets down to is that to be 
considered intelligent you have to do well on 
these tests. In fact, you can't gain access to 
higher education, as I'll argue again later, 
without doing well on such tests. 

In the 1970s a group of us, who call our- 
selves cognitive psychologists, began to feel 
that this definition was somehow wrong, that 
we were missing something in terms of under- 
standing what intelligence really is. Our con- 
clusion (people like Earl Hunt, Jack Carroll, 
Jim Pelegrino, Bob Glaser, and myself} was 
that what was missing in terms of under- 
standing intelligence was an understanding of 
the mental processes that underlie intel- 
ligence. In other words, the tests can give you 
a score, but what they don't give you is an un- 
derstanding of the mental processes that un- 
derlie the score. What we wanted to find was 
what these mental processes are? Why would 
we want to know what the mental processes 
are? 

What I'd like to argue is that for practical 
as well as theoretical reasons, you really ought 
to understand how the person went about 
solving the problems. Let me give you a 
couple of examples. All of you are familiar 
with the kind of problem that appears on in- 
telligence tests, namely verbal analogies, A is 
to B as C is to D. In an example of a verbal 
analogy you cnuld have: "four score and seven 
years ago" is to Lincoln as "I'm not a crook H is 



to, and multiple choice answer options might 
be...Al Capone, Loren Harding, Richard 
Nixon and John Mitchell. The idea is you 
have to see the higher one in the relation, the 
one that relates the first half to the second 
half. If you think about that verbal analogy, 
you realize right away that in orde; to get it 
right you not only have to be a decent 
reasoner, but have to know something about 
American history, both the 1800s and recent 
American history. It turns out that if you look 
at the verbal analogies on most of the intel- 
ligence tests, it's the same. In order to get the 
analogies right on the verbal SAT, for ex- 
ample, or GRE, you really have to know a lot 
of vocabulary. You are more likely to get the 
analogy wrong because you don't know what 
the word; mean than because you don't know 
how to read some of the terms. To do well on 
the Miller Analogies Test, which is 1G0 
analogies, you not only have to be a good 
reasoner but you have to have a good 
vocabulary; you have to know something 
about Greek mythology, math formulas, 
physics formulas, fine wines, and so forth. 
That has an implication. The implication is 
that if you don't have a good score on this 
test, it coulc mean that youVe not a good 
reasoner, but it could also mean you don'* 
have the background information. You don't 
know about fine wines, Greek mythology, 
math formulas, physics formulas, or whatever. 

If you're screening people with diverse 
backgrounds, you really want to know whether 
they did not do well on the test because they 
don't have the background or because they 
can't reason well. The diagnostic implications 
are totally different. All you have to do is give 
the Abraham Lincoln/ Richard Nixon analogy 
in a foreign country and no one knows what 
you're talking about. It's not that they're 
stupid; it's just tha,. they don't have that par- 
ticular backg ound. When you look at verbal 
analogies or other kinds of verba', items, on 
virtually any of these tests, you do not really 
know what you're measuring and you can 
make incorrect predictive and diagnostic in- 
ferences. 

Let's take a completely different domain, 
spatial relations. A lot of intelligence tests will 



have spatial relations questions in which a 
figure is given and you have to say which of 
the figures in another set can be rotated into 
congruence with the given figure. In otter 
words, sometimes they're mirror images, 
sometimes they're not. You have to be able to 
distinguish the mirror images from the ones 
that are just rotation. That kind of item is 
used on the SRA Primary Mental Abilities 
and many other intelligence r , ,ts. 

The reason that kind of problem is of in- 
terest to me is that I used to do very poorly 
on these tests; I have never been good at spa- 
tial relations. Two weeks ago I was in Israel 
and was supposed to go to someone's house 
which was five minutes from the hotel. It took 
me an hour and 15 minutes to f:nd his house. 
What was really bad is that I had a map. Spa- 
tial location tasks are something that I find 
very hard. Predictably, I did quite poorly on 
them in elementary school, but in high school 
I found I could drastically increase my score. I 
realized that I could solve a lot cf these 
problems verbally; I didn't even do the mental 
rotation. For example, you might say that this 
looks like a triangle with a dot in the middle, 
and you start talking your way thiough the 
problem. You can't do it for every spatial 
relations problem but you can do it for 
enough of them so that you can increase your 
scores substantirilly if you happen to be higher 
verbal than you are spatial. So, you could 
have two people taking a spatial relations test, 
one of whom solves the problems verbally 
while the other solves them spatially. They 
could have the same score, but the two scores 
would not mean the same thing at all. If you 
were in job placement, you would want to 
know whether the person was solving the 
problems verbally or spatially, especially if you 
were picking pilots or certain kinds of 
mechanics who may be able to solve the 
problems verbally on a test, but who would 
not be able to do it in the air when respon- 
sible for 400 passengers. So again, unless you 
understand the mental processes that underlie 
the test scores, you can draw conclusions that 
are wrohg. 

For a number of years we did these 
process analyses of intelligence test scores. 



But instead of having just an IQ score, we 
knew what processes the person used to solve 
the problem? what strategies they combined 
those processes into, how they represented 
that information, how long each of the proces- 
ses took, and so forth. We really had a good 
idea of the information processing underlying 
the test. For several years I thought that this 
was a big deal; then I came to the conclusion 
that it was not tha* big a deal at all. We were 
still missing something fundamental in terms 
of understanding intelligence. I came to that 
conclusion by looking at what was actually 
happening in our graduate program. I had 
been director of graduate studies in psychol- 
ogy for a number of years so I was very in- 
volved with the testing program. One of the 
things I saw happening again and again was 
that we would admit people with very, very 
high GRE scores. But when they got into the 
program I would be convinced that they had 
either cheated on the test, had someone else 
take it for them, or had brain damage over 
the summer. The person who came into the 
program did not seem to be the person who 
had taken the test, and their performance just 
was not what we thought it would be. There 
were enough such cases that I became con- 
cerned. 

You can say, maybe it's motivational. 
After all, intelligence doesn't account for 
everything. Of course it doesn't; we do get 
people with motivational problems. But a lot 
of these students seemed to be trying; they 
just did not seem real smart when it got down 
to using whatever the tests were measuring. 
The question became: Why is it that these 
people test well, but when you hire ^em, 
they're duds? That was the beginning oi my 
thinking about the triarchic theory and the 
way the triarchic theory really came about. 

I'm going to say a little bit more about 
three students because to understand their 
pattern of abilities is to understand my theory. 

The three students are Alice, Barbara, and 
Celia. I've changed their names, but they are 
real students 

Alice-Type Learners 
Alice is a student like the first boy in the 
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forest story. She is somebody who came into 
the graduate program with all the right 
credentials. Her GRE scores were over 800, 
she had a 3.98 academic average, and her let- 
ters of recommendation were excellent. 
Everything about her on paper was just su- 
perb and* of course, we admitted 'iter. She was 
our top pick that year. Well, the moral of the 
story is not that Alice was a bomb, she wasn't 
really bad. To the contrary, during her first 
couple of years she was great. She was really 
great at exactly what it waft that the test 
measured. If you gave her a multiple choice 
test she did very well on ft. If you had her 
write a typical academic paper where you 
summarize what other people say and at the 
end have a so-called "creative paragraph," she 
did very well. If I wrote a paper and wanted 
someone to tear i* to shreds, I'd give it to 
Alice. She'd tear it to shreds-and that was 
before Ollie North made that a national pas- 
time. She was very sharp analytically. She 
could take an argument and find any flaws 
that existed in it. Her memory skills were 
good; her analytical skills were good, she was 
what the test said she would be. During thtf 
first couple of years she was one of the top 
two students in the program. 

The interesting thing about Alice is that 
by the time she was done she was roughly in 
the 30th pereentile-the bottom third of the 
program. The question is: What happened to 
Alice? Why did someone who started off so 
well end up poorly? Well, what happened is 
very simple. She was somebody who was very 
good at remembering things and at analyzing 
things, but she was pretty bad at synthesizing 
things or coming up with her own ideas. What 
I want to argue L that the skills Alice lacked 
are not only needed for success in our 
program, they are needed for success in al- 
most any kind of real-world life endeavor. 
What Alice lacked is important for real-world 
intelligence but something tests in general do 
not measure. Take any area of science at all, 
they all have one thing in common; namely, 
what you actually do with the field test has 
next to nothing to do with what you do in 
school in that field. What you do in a typical 
school course is memorize a text book or 



solve the problems that are given to you. In 
our introductory course you are given an in- 
troductory psych book and you Ye expected to 
remember a lot of the experiments that have 
been done. If it were a physics course, you'd 
be given a whole bunch of problems and be 
asked to solve them. I argue that that has 
nothing to do with what science is. If you look 
at the literatiue on science and the sociology 
of science, what scientists are evaluated on is 
not on whether they memorize a text book. If 
you need a book you get it; you read what you 
need to read. You do not solve problems that 
are given to you. If you have a particular 
problem you can't solve, you get help. The 
main thing that you are evaluated on as a 
scientist is the size, the scope, the originality, 
the importance of the policy you deal with. 
It's much more important to look at the kinds 
of problems a person is facing and the 
creativity they bring to bear in solving tho r e 
problems than it is to look at their memory 
and analysis skills. Experiments of not-so-wdl- 
known scientists are not so afferent from 
those done by more prominent scientists; 
what differs is the size of the pnblem or the 
Importance of- the problem with which they 
are dealing. So it .urn* out th::t what Alic°* 
was good at is not irrelevant to doing well, but 
it is not nearly as important as the test im- 
plies. In fact, what Alice lacked was what was 
really needed to succeed as a scientist. 

What about other fields? Take the 
humanities; take writing. It is one thing to be 
told by a teacher in school, "We are all going 
to write an essay on..." and you go home and 
write y)ur essay on.... It's another thin^ when 
you're actually a writer, whether it's writing 
articles for newspapers, writing novels, writing 
poems, or whatever. It's a different thing 
when you have to come up with your own 
ideas for what you're going to write about and 
figure cut how you're going to make it inter- 
esting. I noticed the biggest difference among 
the reporters I deal with is whether they know 
the questions ask. Their story is going to 
depend largely on whether they know what to 
ask me and whether they can synthesize. 
That's the same thing that matters with a 
scientist, this knowing how to ask the right 
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questions. It's also true in art. What distin- 
guishes the more creative <"itists in the real 
world from the not-so-creative ones is their 
ability to deal with sizable and important 
problems in art. 

You could say, these are all specialized 
occupations-science, writing, and art. So lake 
business. We deal a lot with managers and 
upper-level executives, and one of the things 
we hear again and again is complaints about 
their graduate training. The nature of the 
complaints are twofold: one is about their in- 
terpersonal skills, the other is about coming 
up with creative ideas for new business 
products, for new business services, for in- 
novative marketing. Not only do they not 
know how to do it, but what is worse, they 
don't know what they don't know. They think 
they know a lot, and they're not even open to 
learning what they need to do the job well. So 
the same problem applies in business. There's 
nothing wrong with having skills like Alice 
had, but it is not going to be a substitute for 
what you really need to succeed in business. 

Most of us are concerned with education 
where the same concept applies. It's one thing 
to learn how to teach in an education course; 
it is another to get up in front of a class and 
figure out what you're going to do. All of us 
have known of professors who have PhDs in 
education who were lousy teachers. Somehow 
they cannot actually apply what they learned 
in school. 

Several years ago I was giving a talk to 
education professors at the University of 
Puerto Rico, for the first time probably in 
my life, I encountered a serious classroom 
management problem. They weren't listening; 
they just didn't care about what I had to say. 
So I tried the standard classroom manage- 
ment techniques that you learn in school 
books on classroom management. I tried 
lowering my voice so that they'd have to lower 
theirs to hear what I was saying. But the 
presupposition that they wanted to hear what 
I had to say turned out to be false. It actually 
made it easier for them to hear themselves. 
The second thifcg I tried is nicely telling them 
to be quiet so other people in the room could 
hear. But again I assumed that such a group 
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actually existed. I went through a litany of 
things and nothing worked, and I realized that 
when it really comes down to serious class- 
room management problems, the books don't 
work. What happened is a woman in the 
audience stood up, said something in rapid 
fire Spanish, and after that you could have 
heard a pin drop. I don't know what she said 
because she spoke so fast. It was clear that 
this woman had good creative ideas for class- 
room management, and it wasn't something 
she read in a book! 

This value of creative thinking applies as 
well in upper-level jobs. Read Sylvia 
Scribner's work on milk packers. After start- 
ing off doing a standard job, they creatively 
redefine what they're doing to make themsel- 
ves much more efficient. It is really quite 
general. There is even a thesis on horse racing 
that shows that people use creative and clever 
strategies in trying to pick winners. 

The idea then is that Alice lacks some- 
thing. What she lacks is not measured by the 
tests that we use, but it is important for real- 
world pursuits. It may *k. be that important 
in schooling. 

The question arise 0 . How did she get to 
be this way? She is very good at analyzing and 
remembering but not at synthesizing or creat- 
ing. One explanation is she was born that way. 
There .nay be something to that because 
there is a genetic component to intelligence, 
but I cant to pose not a genetic explanation 
but an environmental one. The reason we find 
that most smart kids are smart like Alice is 
not because they were born that way but be- 
cause we made them that way. Alice is basi- 
"illy an outcome of our school system. If we 
find Alices there in abundance, it's because 
we Sve created them. Why is it that when we 
turn out smart people, we turn them out like 
Alice? I think the mechanism is a very simple 
conditioning mechanism. Basically, students in 
school are rewarded for being like Alice. You 
get good test scores; teachers like you; you 
get good grades; you get in the gifted 
programs; you get into good colleges. Everyth- 
ing goes right in schooling if you are smart 
like Alice. So people learn the lesson--be 
smart like Alice. These Alice-like people 
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move along getting heavily reinforced until 
they get to this transition point which I have 
been talking about. What changes is that 
memory and analytical abilities no longer mat- 
ter as much. Other abilities start to matter 
more. The result is that Alice, instead of 
being continually reinforced for being smart 
the way she is, starts being intermittently or 
periodically reinforced. That turns out to be 
very important to understanding adult intel- 
ligence because we know that the kind of 
reinforcement that most sustains a given pat- 
tern of behavior is intermittent reinforcement. 
And that's what starts happening to Alice. 
The irony is that when ' t demands of life 
change, Alice starts doing even more of what 
she's been doing instead of capitalizing on her 
other abilities. 

Another example is the book, Wfiatever 
Happened to the Quiz Kids. Most of these ex- 
tremely bright kids had IQs over 160 and 
were on radio or TV shows called "The Quiz 
Kids." The interesting thing about the book is 
how boring the adulthoods of these kids were. 
They were not people you would write about 
if you were picking them on the basis of their 
adult accomplishments rather than childhood 
ones. The question is why? The reason, I 
think, is simple. If you look at their stories, 
you see again and again this pattern of over- 
capitalization on IQ-like abilities and it just 
stops working. You probably know people like 
that; they keep trying to use abilities that 
don't work. If you're very upset about some- 
thing and you need someone to comfort you, 
but the one you go to gives you a formal 
analysis of why you do not have a problem 
and why you should not be upset, it is not 
going to help you a whole lot. So, over- 
capitalization on IQ-like abilities can actually 
do a person in. Alice may have more creative 
abilities; she just never finds them. 

Barbara-Type Learners 

Let me move on to Barbara who has a 
very different pattern of abilities. When she 
applied, Barbara's grades were okay, but her 
test scores were absolutely rotten. At the 
same time her letters of recommendation 
were absolutely terrific. You could not have 
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two more disparate sources of information 
than we had about Barbara-horrible test 
scores combined with terrific letters written by 
people we knew and respected. They said, 
"Don't pay attention to the test scores but 
look at her work." I decided to look at her ex- 
perimental work and I thought it was really 
inchefui, creative, and inventive, and that Bar- 
bara ought to be a student in our program. 
The reason she ought to be a student in our 
program is because of what I call the fun- 
damental principle of prediction. It is a prin- 
ciple so completely obvious that it tends to be 
ignored. The fundamental principle of predic- 
tion is that the best predictor of behavior in 
the future is the same kind of behavior in the 
past. It applies in schooling and it applies in 
personal relationships. The best predictor of 
what people are going to do is what they're 
doing now. The best predictor of what they 
are going to say is what they say now. 

The problem with Barbara is that she 
probably has not been very heavily reinforced 
for what she docs well. If you look at the dif- 
ference between Alice and Barbara, Alice was 
continually reinforced for what she did well 
and then started being intermittently rein- 
forced. Barbara probably has not been very 
well reinforced because her test scores are not 
good. The question is, what's going to happen 
to her in the future? 

I want to say a bit more about intermit- 
tent reinforcement because I think it is so im- 
portant to intelligence in every aspect of life. 
An example I'd like to give is from work I do 
on interpersonal relationships. If you are in a 
personal relationship and it is not working, 
what do you do? It has been bad for 20 years 
and you've tried everything. What do you do? 
As we used to say in the 60's, you decide to 
dump the person; or as we say in the 80's, you 
decide to restructure the relationship. Now, 
because you see yourself as a nice person, 
you're not mean to the other person. You try 
to be nice and helpful and you hope that she 
will get over you. But you notice that after a 
year or two she is still not over you. The 
reason she is not over you is because you are 
intermittently reinforcing her. You are telling 
her that the relationship is over, yet you are 
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being nice to her anJ secretly giving her hope 
that you'll take her back. You can keep doing 
this for indefinite amounts of time through in- 
termittent reinforcement. Once it starts, it is 
really hard to change its effects. 

But Barbara has never been very heavily 
reinforced for what she does. This is a chance 
to change that. Let's reinforce her for being 
creative and admit her to our program despite 
the fact that she has rotten test scores. 

So I went into the admissions commiUee 
meeting expecting to convince people of this. 
First, Pm fairly persuasive, and I was sup- 
posed to know about testing, and most impor- 
tantly, I was the director of graduate studies. 
Barbara's case was discussed at some length. 
The vote was five to or.e against her admis- 
sion. Now the question is: Why didn't Barbara 
get admitted? She had already shown she 
could do the work that the program requires. 
Given that she can do the criteria and be- 
havior, why is she rejected on the basis of an 
imperfect predictor of the criteria itself? The 
interesting thing, incidentally, about getting 
low test scores is that once you have the low 
quantitative data, everything else looks slightly 
questionable. Why doesn't Barbara get into 
psychology or anywhere else either, because 
she is not going to get into medical school, 
into law school, into business school, or, basi- 
cally, into any access routes to adult occupa- 
tions because they all require the same test or 
a very slight variance of the same test? Now, 
there are those who would argue that overuse 
of tests is just completely irrational. What I 
want to argue is that actually there are ration- 
al reasons why Barbaras don't get in. 

The first reason that selection systems 
work so heavily against Barbaras is what I call 
the culpability reason. What does that mean? 
Suppose you admit Alice and she's a bomb; 
who do you blame? You blame ETS or some- 
one like that. Remember, her test scores were 
good and her whole appearance on paper was 
great; if she doesn't work out no one can 
blame you as an admissions officer. If she 
doesn't work out it certainly isn't your fault. 
But suppose Barbara doesn't work out (and 
some Barbaras won't)? Whose fault is it going 
to be? The answer is...yours! People are going 



ERLC 



to say: "You should have known she would be 
no good; look at those test scores. It's not her 
fault she didn't do well on the program; she 
shouldn't have been admitted in the first 
place." So if you take a Barbara you're taking 
a risk and the risk is that you, rather than 
Barbara, will look bad. 

The second reason Barbaras don't get in 
is what I call the resemblance reason. Ask 
yourself: Who is making the selection and 
placement decisions? The answer is...Alice, or 
people like Alice. You know that because, if 
they're making the decisions, you know they 
got in. If they got in, their test scores had to 
have been pretty good. We tend to judge 
other people by ourselves. Look, I must be 
pretty good because I'm making the decisions 
here, and I did okay on these tests. Why take 
someone like Barbara when there are plenty 
of Alices to go around. So you take people 
who are more like yourself and you per- 
petuate the system. 

A third reason that Barbaras don't get in 
is what you might call the publication reason. 
It's also a very simple one. Places publish 
their scores, or if they don't, someone else 
will. Whether it is Barrons, the American 
Council on Education, or your own university, 
somewhere somebody's going to publish these 
scores. What will happen if you start taking 
Barbaras? Your program will look bad. A 
ccuple of Barbaras can do bad things for your 
average test scores. People will see the 
average test scores go down and they will say, 
"That place is really falling apart; they can't 
even get good students anymore." This is not 
just at the college level. In Connecticut, the 
newspapers publish the statewide mastery test 
scores and I was appalled when I saw that the 
district my kids were in does not have very 
good scores. What am I paying all these taxes 
for if that's the best that the district can 
produce? Besides, it's bad for property values. 
Who's going to want to buy your house if 
you've got schools that are produring those 
kinds of test scores? 

Probably the reason that is most powerful 
is what I call the "rain dance reason." This is 
the way it works. I think these Montana 
mountains are great, and it's really beautiful 
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here, and I was actually thinking last night 
seriously about buying land here. Then 1 
started asking myself when would I ever get 
back? You know, Pm too busy and, besides, 
it's expensive to travel to Montana. What I 
need is a system to keep having my plane fare 
paid. The problem is you stay here two days 
and give your talk on intelligence, and by the 
time you are done they figure: he's told us all 
he knows; let's get someone else next year. So 
you realize that if you want to come back you 
have to do something else, and I doubt if 
peop'^ will hire me to talk about my love life. 
But I noticed there are some drought 
problems arouud here and there are fires. So 
I say, "Hey, look, what I can do is handle 
droughts. If you pay for me to come back, I'll 
make it rain and you won't have to worry 
about all this fire stuff." So you figure, why 
not, especially because I say I'll give you 
double your money back if I can't make it 
rain. You pay for me to come up and I §;et up 
on this very stage and, instead of talking 
about intelligence, I do a rain dance. Doe:> it 
rain? Of course not. So you say you want 
double your money back. But I say, "You must 
be kidding! You know this is one of the worst 
droughts we've had. Yellowstone is practically 
gone. You can't seriously believe that one rain 
dance is going to be sufficient." So I keep 
doing the rain dance. It doesn't take that long 
and I spend the rest of the time enjoying 
myself. After I've done the rain dance for 
awhile, it rains. I say, "Thank you very much. 
It was a great trip. I'm glad I could help out." 

A lot of us don't like to admit that we 
have superstitions, but I think we all have 
them. What does this have to do with Alice 
and Barbara? What I want to argue is that 
testing works the same way in our institutions. 
Virtually all people who make admif-sion 
decisions have some score below which they 
won't admit people. They may not admit that, 
but they do have a score. Here is the interest- 
ing thing. You won't ever see anybody with 
scores below that who has successfully com- 
pleted work in their program! So they always 
get confirmation of what they already believe. 
The reason is that they never take the people 
with scores below that point so they never 
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find out if they can do the work. Superstitions 
are superstitions because you don't disconfirm 
them. That, I argue, is what happens in 
schools because we generally don't take the 
risk to find out if the people can succeed. 

One other thing about the use of testing 
tnat bothers me. I want to tell you about a 
personal experience with my son, Seth. Seth 
was in the second grade where he was in the 
top reading group. After my wife and I split 
up, she moved him to another school district 
which was comparable in socio-economic level 
in every way. They put him in the bottom 
reading group. My question is how could he 
go from the top reading group of one district 
to the bottom in another district? The answer 
is very simple; he didn't do well on the read- 
ing test. Maybe that's not so surprising. It is 
conceivable that he might have a little anxiety 
moving to a new home, a new school district, 
new friends. Anyway, he did not do well on 
the reading test so they put him in the bottom 
group. After a short time, they noticed he was 
reading better than others in the bottom 
group. Nothing surprising about that; it was 
the wrong group. So what did they do? They 
gave him another test! He performs the 
criteria behavior fine, so do they move him up 
to a better reading group? They insist they 
need the predictor. So they give him a readrng 
test again, and he does better so they move 
him up to the second group. He is in the 
second reading group for quite sometime, and 
they notice he is reading better than the 
others. So what do they do? They don't say, 
let's move him to the top reading group; they 
say, let's give him another reading test. So 
they give him another reading test and he 
scores at the level of the first group. What do 
they do? Keep him where he is. I asked them 
why they changed their logic. They told me, 
"Well, it's true that he scored at the level of 
the top group, but now he is a full book be- 
hind the top group. If we move him to the top 
group, he would miss all that's in that book, 
so we're leaving him in the stcond group." So 
I said, "We'll take the reading book home and 
work with him on that book." "No, we don't 
let reading books go home." You sometimes 
wonder whether tests are self-fulfilling 
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prophesies. 

I don't think it's just tests. When I look 
back on my school days, I was very test 
anxious. I did poorly on tests and my teachers 
had very low expectations for me. I wanted to 
please the teachers, so I didn't do veiy well in 
school; I did exactly what they expected of 
me. An odd thing happened. I was doing so 
badly on tests that in 6th grade they sent me 
back to retake the 5th grade test. At first, that 
was embarrassing. But it actually was a god- 
send because I wasn't nervous competing with 
5th graders. After that, I was never test 
anxious again. I started doing better in school 
so teachers raised their expectations. Because 
I wanted to please them, I did better in 
school. Self-fulfilling prophesies are not just 
an abstraction. It happened to me and it hap- 
pened to my son. 

Let me tell you what happened to Bar- 
bara. I figured that someone like Barbara 
should have a chance to ; "ow what she could 
do, so I hired her as a research associate. She 
turned out to be terrific. There were good 
reasons to hire her. People should have a 
chance to show what they can do. Besides, I 
was a little bit over 30, and you know what 
adult educators say. Your fluid ab ;, ity--how 
flexibly, how quickly, how easily you think- 
goes up until about 30 and then it starts to go 
down. I figured if I was not going to have 
good ideas, I ought to have people like Bar- 
bara around. Anyway, Barbara was admitted 
to the pKjram the next year, and she has 
done absolutely well. 

Celia-Type Learners 

Celia was different from Alice and Bar- 
bara. Her grades were good, but not great; 
her test scores were good, but not great; her 
transcripts, her letters of recommendation, 
her extra- curricular activities, everything 
about her was good, but not great. I figured 
this was someone we ought to admit. We 
know she is going to be good, but not great. 
You have to take people like Celia into your 
program; you have no option. There have to 
be people in the program who are good, but 
not great. Abilities are supposed to be dis- 
tributed on the normal curve. You have good 
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people like Celia for statistical reasons so you 
have a distribution. We did, and, sure enough, 
she turned out to be good, hut not great. 

The .arresting thing about Ceiia is that 
when she applied for jobs, despite the fact her 
performance \va<o good but not great, she real- 
ly did well on getting job interviews. Everyone 
wanted to hire her. What was it that Celia 
was doing that made her very attractive on 
the joL market? The answer is very simple: 
she can go into an environment, figure out 
what is needed to move ahead in that en- 
vironment, and do it. She is like the second 
boy in the forest story who was very sharp- 
practically intelligent. In graduate school, she 
figured out what kinds of research are 
rewarded, what journals to publish in, how to 
behave with professors, and so on. And she 
could implement what was needed to be very 
successful. My argument is that is very impor- 
tant. 

We had a student apply who was very dif- 
ferent from Celia. He was both analytically 
and synthetically brilliant. Some professors 
said he was going to be very successful as a 
psychologist, but my reaction was that he was 
going 'o be a flop. The reason I disagreed was 
not that I questioned his academic brilliance 
or his creativity. He had those, but he was in- 
credibly arrogant. He was smart and he knew 
it. You might say that that has nothing to do 
with intelligence, and I would agree that ar- 
rogance is a personality trait, but there is in- 
telligence involved here. The guy was not 
practically smart enough to hide the- fact for 
one day that he was arrogant. To get a job he 
needed to hide it at the job interview. Sure 
enough, he goes to lots of interviews, but no 
one wants to hire him. 

Most people accept that Alice-type be- 
havior is intelligence. Some would accept that 
there's really something to the Barbara- like 
activity. But most feel that Celia is not intel- 
ligent. They believe it is personality or 
manipulativeness or common sense, but it is 
not intelligence. I want to make the argument 
that what Celia has is really intelligence. It is 
important to realize that what we generally 
don't think of as intelligence is so important 
and is really intelligence. We have to look at 
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people in context to understand how smart 
they are. If you don't understand the context, 
you don't understand the intelligence. 

In a study Joe Glick did among a tribe 
called the Kpelle in Liberia, he used a sorting 
task. He gave them names of fruits, such as 
apple, orange, and grapefruit; names of 
vegetables, such as lettuce, broccoli, and 
celery; names of vehicles, such as rafts, bus- 
ses, and boats. They had to sort them into 
piles however they wanted. What he found is 
that the Kpelles did "dumb" things. The 
"smart" thing is to sort taxonomlcally by 
names of fruits, vegetables, or vehicles. Well, 
the Kpelles sorted the functional way. They 
sorted apple with eat or ca" with gas. Instead 
of sorting by higher order categories, ti:ey sort 
by what the thing does. That's not only con- 
sidered stupid by some, it's considered stupid 
in an IQ test. On the Stanford Benet or the 
Wechsler Intelligence Test, defining an 
automobile as a vehicle of conveyance is a 
two point answer. It's a higher order 
taxonomic definition. Saying a car is some- 
thing that goes on a highway or uses gas is a 
one point definition. Functionals are worth 
less; they are not considered as smart. Glick 
was persistent and said, Try sorting a dif- 
ferent way." They sorted a different way, but 
it was still functional. Finally, about to give up 
on the experiment, he said to them, "Why 
don't you sort it the way a fool would"? Guess 
what? They sorted taxonomically! There is a 
good reason for this. If you think about ap- 
ples, what do you think about? You probably 
think about eating, not about it as a fruit or 
an organic substance. If you think about a car, 
you probably think about driving or getting 
gas. You don't think of it as a vehicle of con- 
veyance. In the everyday world, you usually 
think about using things. In school, especially 
when taking tests, that is not the answer you 
are supposed to give. But the Kpelles are not 
an IQ tested culture, so what they did on the 
test is what they do in the everyday world- 
just as you would. They got it wrong, but you 
know from your socialization that is not the 
answer that's expected. If you don't under- 
stand the context, you don't understand what 
people's abilities are. 



A second example is from work done by 
Shirley Heath, an anthropologist who is an ex- 
pert on the development of languages. I want 
to argue again that you cannot understand in- 
telligence if you do not understand the en- 
vironment in which it has been socialized. In- 
telligence to a large extent is socialized. 
Parents want their children to be smart so 
they raise them to be smart. But if what they 
think is smart does not match with what the 
school thinks is smart, the kids look dumb. 
That applies at any level. If what you think is 
smart on a job does not correspond to what 
the workers' abilities are, chances are you will 
say: 'They're not very smart. What are they 
doing in this kind of a job"? Heath found that 
in the three communities she studied, the 
socializations were very different. For ex- 
ample, the children from the lower class black 
community tract, did poorly in school from 
day one. By the time they finish school, they 
are doing absolutely terrible, and the jobs they 
get are the dregs that are available. No one 
wants to hire them because they look so 
stupid. Well, one of the th ngs she discovered 
is that in such tracts, the primary mode of 
communication is nonverbal. Literally, from 
the day the kids are born, there is a lot of em- 
phasis on decoding and encoding of nonverbal 
usage. They are really very sophisticated in 
nonverbal communication. In contrast, the 
children that grow up in the middle-cla$s 
white community become very sophisticated 
in the primary mode of communication used 
there, which is verbal, of course. They learn 
relatively little nonverbal communication. The 
punch line is that if you take a person from a 
verbal community and put that child or adult 
in a tract, that person really looks stupid be- 
cause they cannot communicate with the 
people in the tract. But you know what they 
are going to say: they just cannot communi- 
cate because the people in the tract are dumb. 
The problem is that the school is not indif- 
ferent to which of the environments the per- 
son comes from. The school emphasizes ver- 
bal communication. So then the kids from 
tracting truly look dumb. 

Now the interesting thing is that one could 
make a good argument that we emphasize in 
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our society ?s the importance of nonverbal 
communication. If anything, we ought to be 
rewarding what the parents in the tracts are 
emphasizing. For example take teaching or 
giving a talk. Think of speakers you have 
heard or teachers you had in college. Some 
were very boring! Sometimes I wonder how 
someone could have been boring for 30 years. 
Or did you ever hear a talk by someone very 
senior in the field, but you can't read anything 
on the transparencies? You know he has been 
doing that for 30 years. Why do such people 
make the same mistakes year after year? The 
answer, I think, is quite simple. No one tells 
you that you are incredibly boring. It's too 
risky. The communication you get about being 
boring or that no one can read the slides is a 
giggle or laugh or stirrings in the room. The 
communication is nonverbal. If you are sensi- 
tive to that, you have a chance to improve 
your act. If you don't pick it up, you are going 
to be doing the same dumb thing for the next 
20 years. 

A second example, is any kind of inter- 
view. If you're not doing well in the interview, 
chances are the interviewer is not going to tell 
you, "I want to be honest with you. This is a 
boring interview." Rather, the signs are non- 
verbal. When we interview job candidates 
from California, and they're no good I some- 
times feel like saying: "Look, we brought you 
out on American Airlines and we'll send you 
back in a Greyhound Bus if you'll just go 
now." I don't do that, I go through the whole 
rigamarole and at the end am glad the day is 
finally over. If the interviewee is sensitive to 
nonverbal communication, that person at least 
has a chance of realizing that things are not 
going well and to change the way he or she is 
structuring the interview. 

A study I did at Yale when I worked in 
undergraduate admissions was a cost benefit 
analysis of admissions offices. A main finding 
was that the interview was essentially worth- 
less in terms of selection. Nevertheless, I ar- 
gued that we continue the interview. The cor- 
relation between how well people do in the 
interview and how well they think they have 
done is quite low; they rate themselves higher 
than others rate them. The result is that 



people like interviews even though that is 
based on nonverbal information. Those sensi- 
tive to nonverbal communication have a 
chance of picking up what goes wrong. 

Another important example is from inter- 
personal relationships. Real world intelligence 
applies as much to relationships as it does to 
jobs. When things start to go wrong in inter- 
personal relationships, the communication ini- 
tially is never verbal. The reason it is never 
verbal is that when things start to go wrong, 
you don't know it consciously. Something 
starts to bother you and you're not sure what 
it is. At first you don't even know something 
bothers you, but you start saying nonverbally 
that things are wrong. There are all kinds of 
nonverbal cues that something is bothering 
you, but usually there is nothing verbal. What 
is even more upsetting is that after you realize 
there is a problem, the communication is still 
nonverbal. You know that something is wrong 
but you cannot quite put your finger on what 
is bothering you, so you continue to say noth- 
ing verbally. 

With someone who is sensitive to nonver- 
bal cues, there is a good chance that you can 
nip the problem in the bud. With someone 
not sensitive to nonverbal cues, by the time 
the problem is recognized, it may be too late. 
The same thing applies on the job. When your 
boss starts getting unhappy with you, the cues 
are nonverbal. If you cannot pick them up 
until the boss tells you something is wrong, 
you are in trouble. 

The argument I have been trying to make 
is that this matters. Practical intelligence is 
practical intelligence. It is not trivial; we have 
to pay attention to it. 

Measurement of Intelligence 

We measure best the Alice-type intel- 
ligence. One can argue, we may not do well 
with Barbara-like stuff, but at least we do well 
with Alice-like stuff. What I want to argue is 
that we do not do that well with Alice-type in- 
telligence either, but that there is tremendous 
room for improvement. Let me give you .some 
examples of why it is that, even when we 
measure supposedly well, we do no\ measure 
that well. 
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Start with an important aspect of intel- 
ligence which is in the Alice-like part of my 
theory; that is, the ability to define problems. 
On a test, if you do not define the problem 
the way the examiner defines it, you are in 
trouble. Take an example from my own re- 
search. We did a study of analogical reaso ling 
of 2nd, 4th, and 6th graders and adults. We 
wanted to find out what changes with age. 
Second graders were the worst, 4th graders 
were better, 6th graders were better than that, 
and adults were the best. Basically we showed 
that people get smarter as they get older. But 
I realized that a finding so revolutionary 
would never be accepted by a journal, so I 
decided to look for something less interesting 
in the data. I looked at the data and dis- 
covered something odd. There were 2nd 
graders who got none right, absolutely zero. 
This was odd because we used the two- choice 
analogies, which means a 50% chance of 
being right. You need to realize that in test 
scoring, you never see a zero; what you see is 
a percentile of 1, a grade equivalent of 3, an 
IQ of 10. You don't see that there is some- 
thing really bollixed up. What was wrong in 
my study was very simple. We gave analogies, 
A is to B as C is to D, in a Hebrew day 
school where they had English instruction in 
the morning and Hebrew in the afternoca. 
We did the testing in the afternoon and some 
who got zero circled A or B. Why? They were 
reading right to left. It was in the afternoon 
when they would normally be reading in 
Hebrew so they defined the problems wrong. 
They may have deserved one point off for not 
understanding our very clear directions, but 
they did not deserve zeros. 

Sometimes the people who take the tests 
are smarter than the people who write them. I 
published an article in American Scientist in 
which I gave an example of what I thought 
was a very good item. It was an insight type of 
item called the Water Lily Problem. You have 
water lilies growing on a lake. At the begin- 
ning of the summer, there is one water lily on 
the lake. The water lilies double in area every 
24 hours. At the end of sixty days the lake is 
completely covered with water lilies. On what 
day was it half covered? That is an insight 



problem which you cannot just routinely solve. 
You have to think about it to realize that the 
lake was half covered on the 59th day. Be- 
cause the area doubles every 24 hours, if it 
was completely covered after 60 days, it was 
half covered after 59 days. Voout a month 
and a Iialf after this problem was in the ar- 
ticle, I got a one and a half page, single 
spaced, typed letter from Paul McCreedy, a 
famous aerodynamic designer. He proved that 
if the water lilies really doubled in area every 
24 hours, at the end of 60 days, three-quarters 
of the world would be covered with water 
lilies. The sad thing is that if he had done that 
when he took the test, (a) he would have 
spent all of his time on that one problem, and 
(b) he would have gotten it wrong. Again, 
there is a real danger in terms of how people 
define problems. 

Take that second issue of time allocation, 
another important Alice-like skill. In our cul- 
ture we associate speed with being smart. If 
you say a child or an adult is quick, it is 
another way of saying they are smart. Virtual- 
ly every standardized, group test of intel- 
ligence is timed. Test publishers tell you that 
you don't have to be real fast to get through 
them, but I don't believe them. When I took 
the verbal section of the GRE, I did not 
finish. I was absolutely shocked because by 
that time I had worked at ETS and had 
developed my own intelligence test. I was 
pretty test-wise, yet here I was not finishing a 
test. I had about five problems left and I knew 
I could get a lot of them right if I was given 
the time. That leaves two options; one is to 
get those problems wrong and the other is to 
cheat. When you get the ro'ith section done, 
you go back to the verbal section and finish 
the verbal problems-which is cheating. I am 
not going to tell you what I did, but I will tell 
yju I did very well on the verbal GRE. 

We did a study of lay people's conceptions 
of intelligence in which we asked people in a 
supermarket, a train station, and a library 
what they thought it meant to be smart. They 
said things like, "thinks quickly," "reads quick- 
ly," "talks quickly," and "writes quickly." Quick- 
ness was very important. Our granting agency 
did not like that study; they said people in a 
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supermarket don't know anything. They said 
we should ask experts in the field of intel- 
ligence. So we wrote the experts and got 
responses like H thinks quickly/ 1 "reads quickly," 
"talks quickly," and "writes quickly." It is clear 
that it is not always that fast is smart. Ob- 
viously, you want an air traffic controller to be 
fast. You don't want people who ponder the 
deep meaning of two planes coming at each 
other. If you are taking a test, you want to be 
fast. But there are times when it is not very 
smart. For example, most of us could think of 
some important decision we made in our lives 
and were wrong because we made it too fast. 
We make decisions that are not good because 
we make them too fast. That's not to say that 
it is always smart to be slow. Thurstone ac- 
tually wrote that to be smart is to be able to 
withhold instinctive responses. The smart per- 
son can withhold action, think about it, and 
then decide the smart way to react. All I am 
arguing is that the smart person knows when 
to be fast and when to be slow. 

Ou. research shows that if you look at 
people's planning behavior, for example, 
smart people are not always faster than dumb 
people. Sm' 4 .rt people actually spend more 
time on what is called global planning; that is, 
they spend more time deciding what to do 
and less time planning the way to do it. 
People who are not so smart spend less time 
on global planning. The price they pay is that 
it takes them longer to do the task because 
they have to double back; they get on false 
trails. 

The way tests measure reading com- 
prehension is ecologically ridiculous. You 
could never do in your everyday life what yo, 
do on a reading comprehension test. To do 
well you have to read everything wi*h a fine- 
tooth comb because of the detail questions. If 
you read everything with a fine-'ooth comb in 
your everyday life, you would never get any- 
thing read. You have allocate your time so 
that you read some things carefully and skim 
others. Again, the smart thing in everyday life 
does not pay off on tests. 

Let me move now to Barbara-like intel- 
ligence. If you look at an emerald ring, most 
people will agree that emeralds are gieen. If it 



happened to be a sapphire ring, most would 
agree that sapphires are blue. What I want to 
argue is that it is not clear that the emeralds 
are green or that sapphires are blue. If you 
want to assess Barbara-like intelligence, sug- 
gest to someone that it is possible that 
emeralds are actually grue, meaning green 
until the year 2000 and blue thereafter, 
whereas sapphires are actually bleen, meaning 
blue un.H the year 2000 and green thereafter. 
If you think about it, the evidence is no better 
than emeralds are green than that they are 
grue, because you don't know what's going to 
happen in the year 2000. Now you could say 
that emeralds are green this year, and they 
were green last year, and they were green the 
year before. But they were grue every year 
too. The only year in question is what hap- 
pens in the year 2000. We don't know. You 
could argue that: Why would anyone want a 
complicated concept like grue, which involves 
a switch from green to blue in the year 2000, 
when you could have a simple concept like 
green? What I would argue is that it is not 
clear that grue is not a more complicated cc;i- 
cept than green. Why? Suppose you were 
brought up to believe that emeralds are grue 
and sapphires are bleen. Then you go to some 
crazy place where they tell you that emeralds 
are actually green and sapphires are actually 
blue. Why would you want a complicated con- 
cept like green which involves a switch from 
grue tc bleen when you can have a simple 
concept like grue where the thing is always 
grue? What is more complicated depends on 
where you start. 

For our testing purposes, we were inter- 
ested in a person's ability to think with novel 
kinds of concepts iike grue and bleen or, in 
another scenario, on the planet Kirod, there 
are four kinds of people. Plens, who are born 
young and di^ young; Quests, who are born 
old and die old; Baits, who are born young 
and die old; and Pothens who are born old 
and die young. What you have to do is reason 
with concepts that are not only different from 
those yju know, but different in kind. What 
we found is that Barbara excels Alice in her 
ability to transit between green /blue thinking 
on the one hand, and grue/bleen thinking on 
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the other, and back again. She is comfortable 
making switches that Alice is not comfortable 
making. 

On the Sternberg Multiconventional 
Abilities Test, a test I am working on with 
Psychological Corporation, we give people 
verbal analogies, but the words are simple. It 
is not a vocabulary test, but there is one little 
trick-analogy is preceded with a premise. 
Half the time the premise is ordinary, but half 
the time it is a statement like "sparrows play 
hop scotch" or 'Villains are lovable." It is a 
counterfactual presupposition and you have to 
assume the presupposition is true to solve the 
analogy. It is interesting that Alice will do bet- 
ter than Barbara on a standard verbal anal- 
ogy, but Barbara will do better than Alice if 
you say "imagine" that the world is this way 
rather than the way it is. This is an important 
ability for Barbara-like thinking because what 
we were talking about before was business 
people, scientists, artists, writers, and 
educators who all had in common the Bar- 
bara-like ability to see things in a way that 
other people do not see them. This is the 
ability to look at an old problem in a new way 
or a new problem in an old way, but to see it 
from a perspective that just does not presun- 
pose what everyone else presupposes. 

There is a danger in Barbara-like ability. 
A lot of research has been done in the last 10 
years on expertise, and what it shows is that 
experts know more than novices and they 
structure their knowledge better. What has in- 
terested me for a number of years, and what 
we are now researching, is what I call the cost 
of expertise; that is, ways in which experts be- 
come worse than novices. I have argued that 
the danger of expertise is that they start to 
know too much. The advantage I have over 
my graduate students, being older and more 
educated, is thai I have all kinds of proce- 
duralized routines. Things that for me are 
automatic, for them are hard. To me, some 
things are proceduralized. If you look at it 
from their position, they have an advantage 
that I don't have. Novices are more flexible 
than experts because they don't have as many 
of the proceduralized routines. The novice has 
the advantage of seeing new perspectives that 



the expert starts to lose. 

We have done research, using the game of 
bridge, where we showed the loss of expertise. 
If you give experts and novices bridge games, 
the experts will do better. If you change the 
surface structure of the game, for example, 
change the name of the suits or their order, it 
does not hurt the experts. They have things so 
proceduralized that they automatically make 
the changes. The novices are hurt by that. But 
if you change the deep structure of the game, 
say, the person with the lowest card starts the 
next round instead of the person with the 
highest card, the novices aren't hurt any more 
by that than by the surface changes. But, 
changing the deep structure seriously hurts 
the experts' ability to play the game. In other 
words, the cost of expertif^ is that you be- 
come so good at what you do that you can't 
see another way to do it. You lose whatever 
Barbara-like ability you once may have had. 
We need to build resources into our lives so 
that we do not lose that kind of flexibility. 

Finally, I want to talk about Celia-type in- 
telligence. Basically, to understand intelligence 
you have to understand a person's ability to 
adapt to context. If you are interested in prac- 
tical intelligence in adulthood, you already 
know thai. The question is, can we measure 
it? It is one thing to say that intelligence has 
adaptations, and another thing to measure it. 
Rick Wagner and I decided to try to measure 
it, and the way we started was to interview 
people who had been nominated as practically 
intelligent in two occupations, business execu- 
tives and college professors. We asked them, 
"What does it take to be practically intelligent 
in your field?" 

They agreed on three things. One, IQ 
does not matter much. That did not surprise 
us because we already knew that the correla- 
tion of IQ with life measures of success is 
about .20. 

The second thing they agreed on was 
more embarrassing, what you learn in 
graduate school or professional school doesn't 
matter very much. I thought about it and real- 
ized that I learned in school nothing of what I 
needed to succeed on my job. For example, to 
be practically intelligent as a faculty member 
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you have to learn how to write a grant. If you 
don't get a gram, you don't get any paper 
clips or any paper, and none of the students 
want to work with you because you don't have 
any money. Eventually you lose your job be- 
cause you did not get any research done. The 
interesting thing is that they did not hire you 
to write grant proposals. There was nothing in 
your contract or in your job interview that 
said this was important, but, if you dc>n't know 
how to do it, you're out of a job. Another 
thing is how do you get a big office? Some 
people have big offices, some people have 
small offices. They never tell you that in 
graduate school. I noticed that the people 
with big offices tended to be people who got 
along better with the chairman. They don't 
teach you how to do that either. It could go 
on and on. If you think about what you need 
to succeed on the job, probably only a small 
fraction of it is what learned in graduate 
school. It's not that what is taught in school is 
wrong, it is just not relevant. The valuable 
ideas you get out of graduate or professional 
school is not what you learned in courses, it is 
what we call passive knowledge. 

That is the third thing all agreed on; it is 
passive knowledge that really counts. What 
Celia is so good at is the ability to go into an 
environment, pick up what you need to adapt 
to that environment, and then do it. That is 
what you get out of graduate school, watching 
people who are role models and internalizing 
the way they do it. Passive knowledge is not 
even verbalized, much less taught. 

We constructed tests of passive knowledge 
and decided what you need on a test of pas- 
sive knowledge is exactly what this whole talk 
has been about-the ability to allocate resour- 
ces. Earlier I talked about allocating resources 
in academic settings; the same thing applies in 
everyday settings. The test gave people this 
type of problem. You're in the second year 
on the job. Here are some things you need to 
get done in the next two months; you do not 
have time to do them all. What are your 
priorities? Or, you are a senior executive, or 
senior college professor, giving advice to a 
junior person in your fielJ. Here are some 
pieces of advice about what leads to greater 
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or lesser success in your company or univer- 
sity. Weight the quality of the advice. 

What we found from the passive 
knowledge tests was, on the average, people 
who have been in the field longer have more 
passive knowledge. No surprise there. But 
what is more interesting is that some people 
with much experience have poor passive 
knowledge and some people with little ex- 
perience have very good passive knowledge. It 
is not actually the experience that matters; it 
is what you learn from the experience. 

I see job advertisements that require x 
number years of experience. I think they are 
silly because there are people in the field for 
30 years who are still boring lecturers, who 
still do not know how to do their jobs or how 
to get along with people. It is what you learn 
from the experience that really matters for 
practical intelligence. 

You may ask, is a passive knowledge test 
just a fancy IQ test? Does it have the same 
concepts as an IQ test but with a little window 
dressing to make it more content o' face 
valid? The answer is no; the correlation of 
passive knowledge with IQ is not significant. 
We are not just measuring tl : 3ame concept 
in a fancy way. 

A third finding is that passive knowledge 
scores predict worldwide criteria twice as well 
as do IQ tests. The correlation is about point 
.40 rather than .20. We can actually predict 
fairly well how well people will do on their 
jobs. We have done it now with college 
professors, salesmen, waitresses, businessmen 
and with college students. Much of what you 
need to succeed in school is not what IQ tests 
measure, but passive knowledge. What does 
the teacher expect in the writing of a paper? 
How do you study for an exam? How do you 
study in math versus how do you study in so- 
cial studies? One of the mistakes I made was 
trying to read a calculus book like I read a 
novel. When I got to the exam I had no idea 
where the questions came from. What we 
have found in our research is that passive 
knowledge predicts college grades as well as 
an SAT-type test does. It really counts in col- 
lege as well as in later life. 

Passive knowledge is critical to your 



ability to adapt, but the ability to adapt is not 
all there is to practical intelligence. Some- 
times it is not smart to adapt. If you were told 
you were going to work for a research- 
oriented institution and you end up teaching 
five courses a semester and being on six com- 
mittees, it probably is not the job for someone 
who really wants research. If you are in a 
relationship and it is absolutely not working, 
the smart thing is not to adapt. If you go to 
work for a computer software company be- 
cause you're interested in developing educa- 
tional software, and you end up in the in- 
dustrial espionage division, the smart thing is 
not to adapt. There are mi .y times when the 
smart thing is to select another environment. 
Part of being smart is knowing when to quit. 
There are some jobs you might like to leave 
but can't. There are some relationships you 
don't like but, for one reason or another, can- 
not leave. 

That leads to the third and important op- 
tion, shaping. Practically intelligent persons 
not only know how to adapt to environments, 
or how to select them, but sometimes shape 
the environment to suit them. Lee Iacocca did 
not go to Chrysler thinking what does it take 
to be a good Chrysler person? He defined it! 
People who are great in art, musr, science, 
business, and education do not just adapt to 
the existing environment; they create environ- 
ments for themselves and for others. 



What shapers have in common is not what 
IQ tests measure. One of the things we have 
found they all have in common is that they're 
good at some things but not at others. In fact, 
there are a lot of things they're not good at. 
One of my surprises in graduate school was to 
find famous professors who were bad at many 
things. It is not what they have in common; it 
is that they are really good at something. It 
may be just one thing, it may be two things or 
three, but there are some things they are real- 
ly good at and there are lots of things they are 
bad at. Here are the critical things: they know 
what they know what they are good at; they 
know what they are bad at; and most impor- 
tantly, they "*alize on whatever it is that 
they're good a, .hey make the most of it and 
use it effectively in their lives, in their careers, 
and in their personal relationships. They ex- 
ploit to the fullest what they do well. They 
compensate for things they don't CJ well. 
They either find other people to do these 
things, or they make it unimportant in their 
work, or tbey make themselves just good 
enough so they can get by. 

If you want to understand intelligence, do 
not look just at a static set of abilities. Look 
at a person's abilities to capitalize on 
strengths and compensate for weaknesses. 
Most of all, understand not only Alice-like in- 
telligence, but Barbara-like and Celia-like in- 
telligence too. 
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Strategics and Learning 

Claire E. Weinstein 



When I was a graduate student studying 
educational nd experimental psychology in 
the early 7Us, many researchers were still 
focusing on the learning of nonsense syllables, 
consonant/vowel/consonant letter strings. 
Much of the new and exciting research was 
examining the learning of words, I am glad to 
report to those of you who might be inter- 
ested thct we've gotten to the level of senten- 
ces and paragraphs, and hopefully we will get 
to whole pages pretty soon! But these were 
the types of things that we studied when I was 
a student, and I brought a samp<e word list 
with me. Please take out a piece of paper. I 
have a list of 12 words and what 1 want to do 
is read these words to you at the rate of one 
word every three seconds. Your task is to 
remember as many of these 12 words as you 
can. You go not have to worry about remem- 
bering the words in order. This is what is 
called a free recall task, not a serial recall 
task. All you have to do is tr> to remember as 
many of the 12 words as you can. I am going 
to read the words to you now, but please do 
not write anything at this time. After I have 
read all of the words, we will wait for 20 
seconds and then I will ask you to write down 
as many of the words as you can recall. Again, 
do not worry about word order; do not worry 
about guessing. Okay, let's begin. Bed, rest, 
tired, night, pillow, yawn, cover, dream, sheet, 
pajamas, slumber, alarm. Do not write any- 
thing for 20 seconds. [Idle chatter for 20 
seconds,] Now write as many of the words as 
you can recall; do not worry about word 
order; do not worry about guessing. [Short 
pause.] Okay, what I would like to do is go 
back over the list and I would like you to raise 
your hand to let me know if you remembered 
the word. Hew many of ycu got "bed"? Sharp 
group! How many of you got "rest"? This is 
good. How many of you got H tired M ? How 
many of you got "sleep"? Put all of your hands 
down; the word "sleep" appears nowhere on 
this word list! 

This list was developed by Ken Higby at 



the University of Utah. The word sleep ap- 
pears nowhere on the list, but I used this list 
to make a point more dramatically than I can 
make if I just say it; that we are active infor- 
mation processors. People are not just passive 
receptacles of incoming information, whether 
it's from the outside world or whether it is 
from our own thought processes. We actively 
process information; we actively generate our 
own learning; we actively transform the infor- 
mation that comes to us. Some of you who 
came up with the word sleep will realize that 
you used it to help organize all the actions, 
concepts, and objects that were represented 
on the list. Others will not even be aware you 
did it. In fact, I used this once and had a fel- 
low from the audience come up to me at the 
break and say, "Look, I know you think you 
didn't say it this time, but you did" We had a 
recording and we had to play it for him; he 
was that certain he heard it. 

The fact that some people process infor- 
mation more effectively than others has lead 
to a lot of research. What are some of the dif- 
ferences between more and less successful 
learners? Bob Sternberg spent a lot of time 
this morning talking about practical aspects of 
intelligence and there's a lot of interest now 
in what we call practical aspects of learning or 
learning for real-world tasks. However, in .his 
talk I am going to concentrate more on 
academic learning tasks. My particular inter- 
est is young people and adults returning to 
some jort of postsecondary setting, whether it 
iS a training setting, a formal community col* 
lege setting, or a college or university setting. 
Since we have discovered that there are some 
consistent differences between more and less 
successful learners that can be changed, we 
can do something to help students learn how 
to learn more effectively. For many years 
learning-to-learn phenomena were treated as 
part of developmental psychology, not the 
psychology of learning. This meant that 
people regarded this primarily as something 
that evolved; that is, that as you grew older 
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you learned more about how to learn and so 
this was part of normal development. There- 
fore, if you didn't know all that you needed to 
about learning-to-learn, you just were not in- 
telligent. Basically it was your problem as an 
individual. We now know that many aspects of 
learning-to-learn strategies and skills are ac- 
quired. Now clearly there are some of us that 
are sharper than others. However, most of us 
are not in imminent danger of using all the 
abilities that we have! 

Today I will present a brief overview of 
the kinds of learning strategies that we have 
identified that differentiate between more and 
less successful learners. I will also give a brief 
introduction to an instrument that we use in 
our learning-to-learn course at the University 
of Texas that helps to give some preassess- 
ment of where students are in this are^i. This 
data provides some information about in- 
dividual student deficiencies and where 
teachers should concentrate their efforts. 

One of the problems in discussing learn- 
ing-to-learn is that students' learning and 
study strategies should fo rr i an integrated sys- 
tem. Par. of the problem in the past was that 
many techniques that came out of the study- 
si JUs area were presented like ^ bag of tricks, 
a collection of techniques without much 
guidance in how to use them or adopt them to 
individual needs and contents. That does not 
work. You need heuristics, you need 
guidelines, you need approaches, just as you 
need problem-solving strategies. You cannot 
teach people how to solve every problem they 
will face in life. So we teach approaches, 
strategies, and guidelines so f :hat when people 
meet a particular instance of a problem, they 
have an idea of how to come up with a solu- 
tion that is more optimal, or more efficient, 
or more effective by some criteria other than 
one based on simple trial and error, 

A system is difficult to talk about because 
a system is greater than the sum of the parts. 
l v s the old Gestalt idea: the whole is greater 
than the sum of its parts. So what we need to 
do is artificially break the system down and 
then put it back together again. Here are four 
categories of learning strategies to consider as 
part of a studying/learning system: (a) com- 



prehension monitoring, (b) knowledge acquisi- 
tion, (c) active study ski^s, and (d) support 
stiategies. I will explain them briefly and then 
give more detail about each one. 

Comprehension monitoring is the execu- 
tive control function in a lot of iearnir.g-to- 
learn and comprehension areas. This implies 
knowing when you know, knowing when you 
don't know, and knowing what to do about it. 
How do you monitor your comprehension or 
your understanding? One common example is 
what happens when you are reading along in a 
book or magazine, and all of sudden you real- 
ize your mind has wandered. Your eyes kept 
going, but your mind wandered. When you 
realize that has occurred, you go back and 
find the place where your attention wandered 
and reread the rent of the material. That is 
monkoring your understanding. Something 
triggered you that you were not understanding 
what you were reading, so you monitored 
your Understanding, realized there was a 
problem, and went back. This is comprehen- 
sion monitoring, the executive control func- 
tion. 

Next are knowledge acquisition strategies 
or the "how" of learning. To learn you need to 
build relationships between what you already 
know and what you are trying Jo learn. How 
do y*yU do that? How do you build these 
bridges? We now know that there are dif- 
ferent methods that effective learners use to 
help make sense out of what they are trying 
to leani-whether it is listening in class, read- 
ing a text, or watching a film. A common ex- 
ample of this is the use of analogies. I once 
Lad a junior high school Biology teacher say, 
H I have some students in my class who under- 
stand *he difference between the arteries and 
veins and how the circulatory system works, 
but a couple of my students, no matter what I 
do, can't get the idea straight. Could you 
help?" So I took one youn 6 fellow and we 
started talking. I made up analogies like they 
were going out of style! I talked about thin 
Vain" people, thick arteries, and the heart 
working so hard. NV .ling. All the kid learned 
was that it was extremely important to me 
that he learn the difference between the 
arteries and veins! He tried as hard as he 




could but nothing worked. On the way back to 
the classroom, I was chatting with hirn and 
asked what his dad did. He said, "My father's 
a contract plumber." We went back to my of- 
fice and we didn't say anything about the 
arteries, veins, or the heart. We talked about 
dirty water, we talked about clean water, we 
talked about cold water pipes, wc talked 
about hot water pipes, we talked about water 
pumps. In moments thi? young man had the 
whole idea of the circulatory system and how 
it worked because he had found some way of 
relating what he knew a great deal about to 
something he did not understand. 

The next category is active study skills. 
For years, study skills have been kind of the 
folk medicine of the learning area. There was 
a ot of truth in there, but like many fclk 
remedies there was also a lot of nonsense as 
well. What is being researched now is what is 
useful in study skills and what is not. Many 
stuay skill ideas were developed during a time 
in psychology when behaviorism was of 
primary importance. Now that cognitive 
psychology has gained so much advocacy, we 
are really looking more at what role the 
learner plays in the teaching/learning act. 
This does not trke away the importance of 
the teacher or of good learning materials, but 
it does mean that instead of looking at 
learners, as an outcome variable, we also look 
at them as part of the input. They are not 
simply the result of what goes on in teaching 
and learning but rather part of the process. 

The second aspect of the active study- 
skills category is the active part. Concepts 
from cognitive psychology and from informa- 
tion processing are being incorporated into 
study skills. For example, when I was in high 
school, I learned about outlining while reading 
James Joyce. The instructor said, "I want you 
to go home tonight and create an outline. Put 
the main idea first then important sub ideas 
under it. You can letter them if you want to. 
The second main idea goes next and so on." I 
doubt she talked for five minutes. I had a lot 
of trouble doing this assignment. I didn't 
know what the main idea of James Joyce was 
and I still don't! I couldn't figure out what I 
was supposed to do. Today, when we talk 



main ideas we talk about things that are so 
simple nobody will miss it. People can con- 
centrate on the process. We go through a lot 
of instruction on how you find main ideas, 
then we talk about the kinds of knowledge 
that exist. How are different kinds of 
knowledge related? We talk about cause-ef- 
fect relationships, about part-whole relation- 
ships, about definitions, principles, examples, 
and so forth. Then we will talk about different 
ways you might depict relationships among 
knowledge. Some of them are hierarchial; 
some of them can be depicted in trees, some 
a branch diagrams, some in cause-effect 
i jnds of charts. Then we talk about how all of 
this can be put together to create different 
kinds of outlines. Some will make sense for 
your skills, for the kind of learning you are 
doing, and for the nature of the content 
material. All of that is done in the context of 
the student's goals for doing it. The task re- 
quirement w ; U affect what kind of main ideas 
and information will be pulled out. You can't 
read text materials as if every line was as im- 
portant as every other line, but that's what a 
lot of students try to do. They don't know how 
to pick out important information because 
they don't understand what reading com- 
prehension is. Active study skills means put- 
ting more active information processing con- 
structs into study skills. 

The last category is support strategies. 
This is an int .esting area. Support strategies 
help to generate and maintain climates for 
learning. Support strategies deal with what are 
often called affective variables, variables such 
as motivation, attention, concentration. There 
are two different categories: external support 
strategies and internal support strategies. 

External support strategies include such 
things as having proper lighting or having the 
papers and books you need for class. I can't 
believe the number of times I have talked to 
instructors who give out a diagram in a biol- 
ogy or mechanics class, and the students come 
* u e next period and never think to bring it 
with them. We used to ignore external sup- 
port strategies in our learning-to-learn course 
at the University of Texas. We don't anymore. 
We have had so many experiences like this 
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fellow who walked in the door and stormed 
around in my office saying, "This course is 
such a waste. I'm in a fraternity, and it's so 
noisy there even you couldn't study!" And he 
went on and on. I was sitting there getting 
pretty miffed, so I stood up to my full 5'2" and 
just said, "Sit down!" And I immediately fol- 
lowed up, "Haven't you ever thought of Ieav- 
ing that fraternity house and just going over 
to the library to study?" That was nothing but 
my way of just getting things turned around. 
But this fellow looked up and said, "Wow! 
Thanks Dr. Weinstein," and left. Never had it 
dawned on him to leave a place that was too 
noisy and go somewhere else to study! 

We no longer ignore extvfnal support 
strategies in our course. We have enough 
evidence to indicate that it is important. We 
don't spend a lot of time on it, but we do 
point out the benefits of having light, books, a 
quiet place to study, a corner of the table to 
work on, etc. ^ 

Internal support strategies relate to the 
way that we help to create ana to maintain in- 
ternal climates for learning. Internal support 
strategies include things like being able to 
focus attention, helping to generate motiva- 
tion and interest, diligence, time management, 
dealing with anxiety-all of those things that 
help us to learn more effectively by creating 
the climate in which learning can take place. 
Let me give you an example of this. We do a 
tremendous amount of assessment of the st 
dents who come „ito our Ieaming-to-Iearn 
course, and we have approximately 36 sections 
a year. So we have about 1,000 students who 
take our course in sections of 25 every year. 
Over the years we've discovered that we have 
several different groups of highly anxious stu- 
dents, but there are two types that are of in- 
terest here. If we teach one group about 
learning strategies and study skills, they are 
no longer highly anxious. They do not know 
how to study; when they find out effe. ' 'e 
ways of doing this, their anxiety lowers. 
Another group that we encounter knows a 
tremendous amount about how to use learn- 
ing strategies, but they are so anxious when 
studying for a test, preparing for an oral 
presentation, or something of that nature, that 
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they panic. They are so worried about their 
performance that they do not use the things 
that they know how to use. These are some of 
the ways that we can look at support 
strategies-controlling those things that may 
inhibit our ability either to learn material or 
to perform when we are asked to demonstrate 
the knowledge and skills we have acquired. 

Concentration is another example. For 
many years we tried to find out what con- 
centration was. I can remember as a child 
hearing everybody say: "Concentrate harder!" 
as though it was something you just did. The 
problem is that concentration is as much a 
resultant of the things we do as a resultant of 
things that we don't do. For example, to help 
yourself concentrate you use things to help 
focus your attention. Forms of elaboration, 
such as creating analogies, are major ways of 
using what we already know to add to what 
we are trying to learn. That keeps you actively 
involved in what you are doing so it helps to 
focus your attention on che task at hand. For 
example, comprehension mon?*oring helps 
focus concentration. 

Examples of things you do not want to do 
are often related to anxiety. Many students 
who are intelligent and who have the needed 
skills often end up having less time to take a 
class exam or other test because of their 
anxiety. It is nc that they don't know how to 
answer the test questions; it is that they are 
so worried about their perfo mance that 
worry interferes with their ability to perform. 
They spend a large part of the test time wor- 
rying about how they are going to do on it 
and less of their time taking the test. That is 
what I mean by saying concentration is a 
resultant of things we do and a resultant of 
the things we do not do. 

Now I would like to cycle back and spend 
a little more time on two of these aieas. The 
first area is comprehensio i monitoring. As I 
said, comprehension monitoring is a kind of 
executive control function; that is, knowing 
when you know, knowing when you don't 
know, and knowing what to do about it. 

To monitor your comprehension you need 
certain kinds of knowledge. One is knowledge 
of yourself as a learner. For example, what 
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are your learning preferences, what subjects 
do you like, what are your best times of day 
for studying? Knowing about our preferences, 
knowing which kinds of tasks are easier or 
harder, helps us allocate resources; it helps us 
allocate time; it helps us know when we need 
help. It helps you know that if you have a sub- 
ject that you are not good at or do not like, 
you might want to participate in tutoring ses- 
sions, go to group reviews, or schedule regular 
meetings with a tutor or teaching assistant. 
So, knowing about your strengths, your weak- 
nesses, your preferences, your better times of 
day, helps you to allocate resources including 
yourself. 

Another kind of knowledge needed is 
knowledge of tasks. If you don't know what a 
task requires, then how can you determine if 
you completed it successfully? It is very im- 
portant for us as teachers to communicate 
what we want students to do, whether it is 
homework assignments or other things, but it 
is also necessary for us to help students to un- 
derstand what different tasks require. By dif- 
ferent tasks, I mean tasks like reading or lis- 
tening, participating in class, or different 
kinds of learning tasks such as discrimination 
learning, concept learning, and principle 
learning. They all require different ways of 
determining whether the task has been met or 
not. Bob Sternberg pointed out one of these 
problems for text reading. If you read your 
textbooks at the detail level you will never get 
through them. Yet many students read a 
textbook cover to cover; but worse, they treat 
every single line as if it were just as important 
as every other. We have interviewed students 
at colleges and found people who were work- 
ing very hard, studying six hours a day, but 
not doing it effectively. In Adult Education 
this is a particular problem. Many students 
entering postsecondary settings had their last 
experience with formal education 10 or 15 
years ago in high school. High school teaching 
is very different from college teaching. It is 
much more teacher directed. In our learning- 
to-learn course it is not unusual for us to get 
young people who were valedictorians or 
salutatorians in ihdt high school class, but 
who are now, a year or two later, experiencing 



incredible difficulty in college. What was ap- 
propriate for high performance in high school 
and what was expected in terms of taking 
responsibility for one's own learning is very 
different in college. 

The third type of knowledge needed for 
comprehension monitoring is knowledge of 
strategies. Strategies include things from all 
these different categories. What are the dif- 
ferent kinds of skills that I can use to help 
myself? What do I know about learning-to- 
learn? What do I know about comprehension 
monitoring and knowledge acquisition? What 
kinds of study skills or support strategies can I 
use to generate climates in which I will more 
likely accomplish the task? Finally, content or 
prior knowledge in the area I am about to 
study will also help me understand what I am 
doing. There are two reasons why prior 
knowledge is so important. In academic or 
professional situati his there are two major 
things that differentiate experts from novices. 
One is fairly obvious; experts know more. But 
research in expert/novice differences has also 
shown that a major difference between ex- 
perts and novices is not just in the amount of 
knowledge that they have; it is also how that 
knowledge is organized. When facing a learn- 
ing task, prior knowledge can help as sort of a 
knowledge acquisition strategy. Recalling what 
you already know about an area often helps 
you make sense out of something new. Prior 
knowledge can help make what you are trying 
to learn meaningful and not just an isolated 
piece of information. 

The second benefit relates to storage and 
to later use. An example to make this point 
clearer is the way two p °ople use filing 
cabinets. Both take new information, put it in 
a folder, label the folder, and put it in the 
cabinet. One just keeps adding things to the 
file; the other tries to put related information 
together. As time goes on, it is easy to see 
who will have the easier time retrieving infor- 
mation-the one with the organized files. That 
is what research points out about organized 
memory. One of the ways to do that is to con- 
sider other knowledge you have about an area 
as you study something new. You are more 
likely to store that information with hooks 
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leading to the other knowledge and thus have 
more ways of getting to it. That is the second 
reason why prior knowledge is helpful. 

One of the things we do with all these dif- 
ferent kinds of knowledge is use them to help 
monitor our comprehension. The basic way 
we do this monitoring is through some form 
of self-assessment. We assess our own learn- 
ing to determine if it is being successful or 
not. A problem that can arise for students is 
something referred to as secondary ignorance- 
-you don't know that you don't know! Many 
times students have the illusion of knowing. 
They say, "I really understand how that works, 
but I can't put it into words." You may have a 
sense that you know it, but rarely do you real- 
ly understand something if you can't com- 
municate it. One form of self-assessment is 
through the use of questions. For a long time 
the only approach we recommended to stu- 
dents was to come up with questions to guide 
reading. In fact, in a lot of the textbooks being 
created today, the summaries and the review 
questions are put at the beginning of the 
chapte rather than at the end. The reason is 
because we know questions help to create 
goals to guide reading and to use for com- 
prehension monitoring. You can check to see 
if you understand as you go along. Today we 
realize there are also other ways to monitor 
comprehension. One way to do this is through 
the use of application. Try to apply a new 
concept or principle. If you have trouble ap- 
plying it, you get some sense of the degree to 
which you understand it. 

Practice, or actually trying to perform the 
task that you need to do, and seeing where 
you make mistakes also provides feedback for 
comprehension monitoring. Another way is 
creative organization or transformation. Try 
to transform the knowledge into another 
form. The earliest levels of understanding are 
often considered to be reflected by a student's 
ability to summarize and paraphrase. If you 
can summarize and paraphrase, that's the first 
level of comprehension. One can repeat 
something verbatim as long as they have 
eaough memory capacity to hold it. It is like 
the little kid who comes und tells you a dirty 
joke. You look at the child and know he has 
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no id^a what he is talking aboui. As soon as a 
learner starts at the level of transformation, 
organization, doing something with 
knowledge, then he has shown some level of 
understanding. 

Communicating information to someone 
else also might help you educate your own un- 
derstanding. When I want to learn a new area, 
I propose teaching a course in it. It is one of 
the ways I learn best. Never have I found 
where the holes in my knowledge exist more 
quickly than when ^ve tried to teach some- 
thin" that I didn't fully understand. In fact, 
one of the most cost-effective strategies that a 
lot of people are using now is something 
called cooperative learning. With adults, 
cooperative learning involves taking advantage 
of the best of peer tutoring. Research shows 
that in terms of learning-to-learn, the person 
doing the tutoring learns far more than the 
person who is being tutored. The person 
doing the tutoring learns about learning-to- 
learn as well as helping to consolidate his or 
her knowledge and integrate it across areas. 
The people being tutored are learning infor- 
mation, but they remain dependent on the 
tutor or someone else to give it to them. They 
are not really learning how to learn it for 
themselves. 

Cooperative learning takes advantage of 
the best aspects of tutoring. For example, let's 
say that you and I are a cooperative learning 
pair. We're in a class together and after we go 
to a lecture I go through the first page of lec- 
ture notes and say here is what I think Dr. 
Smith was saying this morning. Here is the 
first main point that was made and here is the 
second. And you say, "No, I think that was an 
example he used. I think his second point was 
this because...." Notice we begin talking about 
what we did and how we learned this as well 
as getting content transmitted. When we move 
on to the next page of notes, we switch roles. 
Now you are the reciter and I am the criti- 
quer. In this way, both students get the 
benefit of participating in the process. Who's 
the tutor, who's being tutored? Both monitor 
each other. Both get a chance to improve con- 
tent knowledge at the same time that they 
also increase their knowledge about learning- 
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to-learn. So it is an incredibly effective 
strategy and one that is cost effective. It 
doesn't cost anything but a bit of time. I used 
this in one of my courses at UT. We created 
what we call "study buddies." Others have im- 
plemented it for the first 15 minutes of class, 
where pairs go through the textbook or other 
exercises together. It is a very potent techni- 
que. The only warning I would give is that you 
need to demonstrate this process for people 
because students are not used to doing this. 
You can check up on the ^tirs after you make 
them, looking for personality conflicts or 
other things. But aside from monitoring such 
things, it is a very easy process to get going. It 
is a takeoff on what they used to do in profes- 
sional schools and what some students have 
been doing on their own for years. Another 
word of caution though is to be careful of who 
pairs up with whom. If you let students do it, 
often you will find your D and F students love 
working together because they reinforce each 
other's mistakes. So you need to be a bit care- 
ful how pairs are formed. 

A final point in this area is that research 
has shown that it is not as critical how you 
monitor as it is that you monitor. This is a 
very important point and it relates to 
knowledge acquisition strategies as well. The 
fact that you monitor your comprehension has 
a critical impact on how successful you are; 
the particular method you use does not. In 
fact, for the same task, many different 
methods are used by different students. Even 
for what appear to be highly similar tasks, dif- 
ferent methods are used by the same student. 
There are many things that can determine 
how we monitor comprehension. 

For example, things like the nature of the 
assignment or ratigue affect it. The critical 
point is that students need to have a reper- 
toire of different ways that they can monitor 
learning. Those things that work for us day In 
and day out as we do our learning activities 
are fine, but we need other techniques when 
we run into problems. 

This also relates to something we call 
fluency and flexibility of thought. In terms of 
problem solving, fluency can be described as 
the number of solutions or the number of al- 



ternatives in the decision process that a 
learner comes up with. Flexibility is the num- 
ber of categories into which these alternatives 
fit. 

As an example, we can go back again to 
the experience of reading along and discover- 
ing that your eyes have kept moving but your 
mind has not. Earlier I asked you what you 
would do, and you said you would go back 
over the material. What else could you do if 
you realized your attention wandered? Fluen- 
cy and flexibility of thought might lead you to 
suggest taking a break, getting a drink, 
making a phone call, or settling the argument 
that is percolating in your mind. You might go 
over the summary or review the in'.oduction. 
Phoning somebody else in the class might 
help; so might using another source. You 
could wait to ask the instructor about it the 
next day or you could call someone in your 
study group. There's a number of different 
things that could be done. The number of 
things you come up with is a measure of your 
fluency. Flexibility would be the number of 
different categories. Suppose a person came 
up with five different things they could do 
such as take a break, go get a sandwich, go to 
the bathroom, take a walk, call up a friend. 
They have five actions but all are in one 
category-take a break. They would have a 
high fluency score but very low flexibility 
score. Someone else might suggest five things 
but one might be take a break, one call a 
friend in the class, another reread the 
material, another get another source. Notice 
that this person would have the same fluency 
score but a very different flexibility score. 

Effective comprehension monitoring re- 
quires both a degree of fluency and of 
flexibility. However, many students in 
postsecondary educational settings do not 
evidence a great deal of fluency or flexibility 
in their thinking or the learning strategies that 
they use. When you interview them about the 
things they do in typical academic situations, 
often they can think of only one thing to do. 
When you get to two possibilities you've now 
eliminated almost all of the students you deal 
with. Three is such a small minority that 
you're thrilled when you find someone in this 
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group! It is very important to help students 
develop a repertoire of strategies so that they 
have some fluency in what they can resort to 
when they run into an academic problem, and 
so that they have some flexibility so if ttie 
type of solution or approach that they chocse 
doesn't work, they can find another. 

A final aspect of comprehension monitor- 
ing is what is called fix-up strategies. Fix-up 
strategies are what one does to fix com- 
prehension problems. Fluency and flexibility 
of thought are most critical in discovering 
what can be done to solve a problem. You all 
laughed at the example I gave of the young 
man who was upset with our course because 
his fraternity was so noisy no one could ex- 
pect him to study there. He never thought of 
leaving to study someplace else. Unfortunate- 
ly, this is a common problem; students need 
to be more fluent and flexible in their think- 
ing end learning processes. 

The second major area of learning 
strategies I want to discuss a bit more is inter- 
nal support strategies. For a long time we 
have had good ideas about some of the thing? 
that students report as debilitating to school 
performance. One of the most common things 
that decrease the effectiveness of their perfor- 
mance is anxiety. It is difficult to find a stu- 
dent *sl o does not report that at some time 
debilitating anxiety has severely affected his 
or her performance in some academic situa- 
tion. In fact, today the hottest research area 
in anxiety is with elementary school children. 
By second grade students often show trait 
anxiety as a problem. When I was in graduate 
school I had a friend who had a little boy 
named David who was very beguiling. He was 
three years old and the love of David's life 
was preschool. He was so crazy about it that if 
he woke up at four in the morning, he would 
put on his clothes and get ready for school. 
Then one morning David did not want to go 
to preschool. In fact, he was hiding in the 
closet. They had to dress him; they had to 
take him by car to preschool; they had to 
physically hand him to the teacher. Nobody 
could figure out what was going on. That 
night we found out what had happened; David 
came home with a note pinned to him. David 



wasn't sure what he had done wrong but the 
message had very clearly been communicated 
to him that there was a problem. David had 
flunked threes; he didn't do very well in twos 
either. David did not want to go back to the 
preschool. 

The experience of debilitating anxiety is so 
ubiquitous and so personal that we have 
developed a number of interventions to help 
reduce it. When I was a graduate student, 
there were dozens of studies trying to help 
people with their anxiety and to improve their 
study skills and their performance. The inter- 
esting thing was that many of these studies 
used relaxation therapy. In its simplest form 
relaxation involves going through the body 
systematically, starting at one end. You tense 
each muscle group and then relax it. The idea 
is that you need to be aware of tension in 
your muscles and you need to learn when they 
feel calm so that you can then reproduce this 
calmness. It is an extremely effective techni- 
que for many health problems. It is very im- 
portant to learn techniques that help you to 
stay calm. However, from a cognitive 
standpoint and in terms of academic perfor- 
mance, it did not have much impact. The stu- 
dents were calmer, but they were still flunk- 
ing! We were not able to explain anxiety until 
Jerri Wine came up with something called a 
cognitive attentional view of anxiety. It is cog- 
nitive because it focuses on our own thought 
processes and it is attentional because her ex- 
planatory mechanism is in the area of atten- 
tion. The relationship between anxiety and 
performance is a curvilinear relationship. 
Most people do fine at a mid-level of arousal. 
They are at the optimal level for motivation, 
for paying attention. However, foi many 
people performance is seriously affected as 
anxiety continues to increase. 

Now we know there are two dimensions to 
anxiety. One dimension is emotionality-but- 
terflies in the stomach, hives, tick in the eye, 
buckling knees, sweaty palms, all of these dif- 
ferent things. People experience different 
things emotionally in response to anxious 
states. 

The other part of anxiety is worry, some- 
times called cognitive worry. This is the major 
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culprit from the perspective of academic 
learning and performance. Worry is often 
manifested in something we call negative self- 
talk which feeds on itself. Tm no good; I 
can't do it; why am I here?" All of this nega- 
tive self-talk feeds on something called irra- 
tional beliefs such as, "The rest of my life 
depends upon this one task." There's an excel- 
lent book in this area for those interested. I* 
is directed to the intelligent layman and is put 
out by Monarch Press, Stress, Sanity and Sur- 
vival, by Rob Woolfolk and Frank Richardson. 

Let me give you an example of how 
anxiety debilitates performance. We do not 
know what the limits of storage are in human 
memory. Under surgical stimulation people 
have remembered incredible things. We know 
we do have limits to accessibility and we 
definitely have limits to the amount of infor- 
mation we can process at any point in time. 
Novv^this is artificial, but I'm making it up for 
purposes of discussion. Let's say we have 10 
units of processing space we can use. But I'm 
worried about something. Now, I can really 
get into worrying. Pm Jewish and have 4,000 
plus years of history of worrying! Let's say I 
need six units to focus on my worry. If I want 
to do something that is not very demanding 
for me, I can do it while I am worried. When 
I am really worried my house is immaculate; 
it only takes one or two units. I can clean and 
worry with no problem whatsoever. But sup- 
pose the beginning of the semester is ap- 
proaching and I need to be working on lec- 
tures. This takes a lot of concentration; maybe 
this needs eight or nine units. Now I have a 
problem. If Pm prone to high anxiety there is 
competition for processing space. Thus, one 
of the mechanisms by which anxiety disrupts 
performance is by diverting our attention. It 
diverts our attention from task-relevant 
thoughts and behaviors to inelevant thoughts 
and worries. 

We can do many things to help folks who 
wony. Let me give you just one example. In 
one course we tp~.ch something called thought 
stopping. It involves learning to monitor what 
you're thinking and determining whether your 
thinking is task irrelevant. For example, we 
teach our students to just monitor what 



they're thinking every couple of minutes while 
taking a test. If they find they are focused on 
fears and worries, they tell themselves "STOP" 
and focus back on the test. It sounds very 
simplistic, but it really helps. V/e tell students, 
"You can worry; just don't do it during my 
test!" When we first started teaching this we 
had faculty from all over the university calling 
us and asking: "Dr. Weinstein, why peri- 
odically during my lecture do some of your 
students say "STOP," "STOP," "STOP!" We 
never thought to teach internalization...but we 
do now! 

It is very important to discuss irrational 
beliefs with students. When I work with older 
students or women returning to school, I 
never talk immediately about learning 
strategies. I discuss with them what they are 
afraid of about coming back to school. So we 
get through things like, "How am I going to 
tell my kid I got a D on an English essay?" or 
"What's my husband going to think about this 
or that?" People are anxious to talk about ir- 
rational beliefs. You need to be able to iden- 
tify them so that you can substitute more 
realistic self-talk for negative self-talk before 
you get into a vicious cycle. If people think, "I 
can't do it, I'm going to fail," that takes time 
away from both study and performance and, 
sure enough, they do not do well. Then they 
have confirming evidence and get even more 
frightened and anxious. 

There are several ways that people teach 
learning strategies. One mode by which 
people teach learning strategies is direct in- 
struction. These methods range from our 
course at the University of Texas which is a 
regular three-credit, elective course for stu- 
dents at the university, to workshops at learn- 
ing centers, focus groups, and all sorts of 
variants. In all of these the contexts, the con- 
tent of the class focuses on learning and study 
strategies and skills. 

The second method is what v/e call the 
metacurriculum. This is when instructors 
teach learning-to-learn as part of their regular 
content teaching. The metacurriculum can be 
done in either a planned or an unplanned 
way, or a combination of both. 

A planned approach involves identifying 



25 



certain times in the semester to spend time 
talking about learnings-learn. For instance, 
the beginning ot the semester is an excellent 
time to talk about how to read a textbook. 
When you assign a long-term project, it is a 
wonderful time to talk about time manage- 
ment. We have discovered that a better time 
to talk about tests is after, rather than before, 
the first test. People are much more inter- 
ested in what you have to say. Before the first 
test, by the way, is a wonderful time to talk 
about anxiety and its effects on performance 
and techniques such as thought stopping. 

Unplanned times to implement the 
metacurriculum arc those times when things 
happen in class on which you can capitalize. 
For example, you are teaching something and 
you come up with an excellent analogy that 
really works for your students. That would be 
a terrific time to talk about what an analogy 
is, how it helps students, and how they can 
use analogies on their own. Also, talk about 
transfer, how th^.y can use the skill in other 
situations. W. >u assign practice exercises, 
tell students wny you are doing it. Interviews 
show students think teachers use practice ex- 
ercises as busy work. They have no idea that 
you may have any other reason for doing it. 

One majV problem in this area is assess- 
ment. How do you determine what your stu- 
dents do or do not know about learning 
strategies that are related to successful perfor- 
mance? We have concentrated part of our ef- 
forts on the area of assessment. When we 
began our course, we had nothing that could 
measure prior learning-to-learn knowledge 
and thus had great difficulty determining 
where students needed individual help. That is 
one reason why we developed the Learning 
and Study Strategies Inventory (LASSI). Basi- 
cally, the LASSI is a 77-item self-report in- 
strument that yields scores in 10 different 
areas that give students an idea of where their 
learning strategy strengths and weaknesses lie. 
It helps instructors place people who need 
some developmental education or develop 
ideas of what to stress in a metacurriculum. 
There are some methods that are far more 
important for math or for history than for 
science. LASSI provides summary sheets so 



people can see their strength or weakness on 
each scale. 

Let me briefly go over the scales with you. 
The first is an Attitude Scale and this assesses 
students' general attitude toward college and 
how it fits with their other goals. Much work 
we do early in our course relates to goal set- 
ting. Many students cannot generate the 
motivation for academic tasks because they 
do not see how they fit in with their goals. 
For many students, the problem is that they 
don't have clear goals. When you get them to 
start talking about goals, they often mention 
things like a red sports car, a big house, a big 
dog. We try to help such students to re-ex- 
amine or to establish goals and then to 
analyze what it will take to meet their goals. 
We also look at the relationship between 
long-term goals and short-term goals. The at- 
titude scale starts to get at these things A 
sample item on it is: I feel confused as to 
what my goals should be. 

The next scale is a more specific one, the 
Motivation Scale. Rather than general inter- 
ests and attitudes toward college, this gets 
down to the nitty gritty of going to school. 
Are you willing to show the diligence that it 
takes to go to school? Are you willing to do 
your homework and keep up with assign- 
ments? Do you go to class? When work is dif- 
ficult, do you give up or only study the easy 
parts? 

The Time Management Scale is fairly 
straightforward but really has two parts. One 
deals with what students know about time 
management; the second part with what they 
do about it. One can know time management 
principles but not actually app 5 them. Ex- 
amples of items are: "I only study when there 
is the pressure of a test," or "When I decide to 
study I set aside a specific length of time." 

The Anxiety Scale fits very closely with 
the concept of anxiety that I discussed earlier 
but concentrates on the worry aspect. Ex- 
amples of items are: "I'm very tense when I 
study,' 1 and "Worrying about doing poorly in- 
terferes with my concentration." 

The Concentration Scale also looks at the 
two aspects we talked about before, that is, 
those things that the students do to help 
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themselves concentrate and those things that 
they do that may distract them. For example, 
"During lectures I think of other things and 
don't really listen to what is being said." That 
is a sample item from this scale. 

The Information Processing Scale has 
many pieces to it. It looks at elaboration but 
also has some comprehension monitoring and 
some reasoning items in it. Actually, it con- 
centrates primarily on elaboration and or- 
ganization strategies. For example, "I translate 
what I am studying into my own words." 

The scale on selecting ideas is fairly 
straightforward. Thk really asks if they can 
pick out the important information for further 
study. 

The Study Aides Scale is another one that 
has two parts. One part assesses whether stu- 
dents know how to use study aides that are 
provided. This includes review sections, mark- 
ings in a textbook, and review sheets. The 
second part asks if they can create study aides 
for themselves. It looks at text marking, un- 
derlining, and creating review sheets. For ex- 
ample, "I use special helps in my textbooks 
such as italics and headings," or "When avail- 
able, I attend group review sessions." 

The Self-Testing Scale reviev : much of 
what we talked about in comprehension 
monitoring, testing yourself, and testing your 
understanding. For example, "I stop peri- 
odically while reading and mentally review 
what has been said." 

Finally, there is a Test Strategy Scale. It 
has it'jms about how people prepare to take 
tests and items about what they do during a 
test. 



If you look at the ins!rument and pull 
away the blue sheet, you will see that the 
student's marks transfer onto the back sheet 
and thus it is self-scoring. The LASSI takes 
approximately 20-25 minutes to complete and 
score (there are also computer- administered 
and scored versions for the APPLE and IBM 
PCs). 

There is much that remains to be done in 
faculty as well as student training in the area 
of learning-to-learn. Many processors will still 
say to me: "Why do we need to bother with 
tb* ? What we need to do is help people learn 
content." I think that today many of you in 
adult edu ation realize the need more than 
other groups of college teachers. People today 
change their jobs so many times that the 
Bureau of Labor Statistics indicates five to 
seven job changes in a lifetime: is not unusual. 
We are really helping to prepare people for 
lifelong learning. It is extremely difficult, as 
college instructors and professors, to justify 
our existence based on the transmission of set 
content in history, or English, or science, or in 
any area. We need to teach content but we 
ne^u also to help students develop the ability 
to manage their own learning. We need to 
help develop the skills and strategies that they 
need to take more responsibility for their own 
learning and to be able to manage it. 

There's an old expression that says, "If you 
give a person a iish, you've fed ihem for a 
day; if you teach them how to fish, you've fed 
them for a lifetime/' I hope that we define at 
least part of our job as college faculty a^ help 
ing people learn how to fish. 
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Reaction to Presentations: A Panel of Adult Education 

Professors 

Huey Long, Sharan Merriam, Burton Sisco, and Chere Coggins 



Following the presentations of Robert 
Sternberg and Claire Weinstein, four mem- 
bers of the audience, all professors of adult 
education, took turns commenting on ideas 
that struck them as they listened to the 
presenters. The purpose was to give focus to 
further discussions of intelligence and learning 
strategies. The four adult education professors 
were: Huey Long, University of Oklahoma; 
Sharan Merriam, University of Georgia; Burt 
Sisco, University of Wyoming; and Chcre 
Coggins, University of Wisconsin. Rodney 
Fulton of the Kellogg Center, Montana State 
University, acted as moderator. 

Huey Long: What I have tried to do is 
take a convergent perspective and see what is 
common to both the topics that were dis- 
cussed today. I've moved from Sternberg's 
ideas of intelligence to Weinstein's learning 
strategies to the realization that one defini- 
tion of intelligence is the ability to learn. 

Both speakers presented what I con- 
sidered to be distinctive ideas and 1 believe it 
is helpful to consider the relationship between 
intelligence and learning ability. Analytically, 
which causes which is not always clear. In fact, 
both presenters provided information that 
suggests that intelligence and learning ability 
interact. The underlying mechanisms of 
genetic, physiological, and sociological aspects 
remain unclear, however. Sternberg's theory 
illustrates a point developed by GilfcH that 
intelligence may be conceptualized as a multi- 
faceted phenomenon. Similarly, learning a;SO 
seems to be composed of a number of sub- 
processes. How these subprocesses are con- 
trolled, how they are evoked, and how they 
are reinforced to yield an Alice, a Barbara, or 
a Celia remain to be elaborated. 

The idea of different kinds of intelligence, 
as suggested by Sternberg, legitimizes what 
adult educators have been suggesting for 
years. Also, Weinstein's emphasis on learning 
to-learn is to be found in the central core of 
most adult educators' philosophies. Certainly 



it does not detract from either of the presen- 
tations to note that similar eas are found in 
the literature of earlier \ s. Sternberg's 
comments on flexibility of indigence recalls 
to rnind Milton Rokeach's, "Joe Doodle Bug 
Problem" and his "Theory of Dogmatism." 
Weinstein's comments are not unlike some of 
the observations of John Havell and Robert 
Gagne concerning metacognition. Perhaps, 
n. ither are these concepts greatly different 
from some of the ideas concerning self- 
directed learning. For example, I am of the 
opinion that the only necessary and sufficient 
cause for self-directed learning is what I call 
psychological control, meaning personal con- 
trol of the executive controlled process men- 
tioned by Weins'ein. 

An important point for adult educators is 
the role of the instructor r teacher. I think 
both speakers indicated that this role is larger 
than the telling and giving of assignments. We 
h?»ve a challenge to provide environments in 
which the adult learner can develop a broader 
array of intelligence and apply a repertoire of 
learning skills. For example, one of the ques- 
tions often asked relates to what William Kil- 
patrick called "three kinds of learning," that is. 
primary learning, associated learning, and at- 
tended learning. Often what we find in 
schooling is an emphasis on primary learning 
with neglect of associated and attendent 
learning. I think this may be what contributes 
to the development of an Alice. If we could 
be a bit more sensitive to the associated and 
attenJent learnings, perhaps we would also be 
more confident in contributing to a well- 
rounded individual that would have the 
strengths of all three of the kinds of people 
that Professor Sternberg illustrated for us. 

I also find the same phenomenon il- 
lustrated in other areas. Ian Mitroffs study of 
Apollo scientists evealed that among these 
scientists there was a clear distribution of 
theae scientists on a kind of reputational poll. 
There were some that were exciting 
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speculators, grand conceptualizes; there were 
some that were in the middle; and there were 
others toward the end of the continuum that 
were extremely limited by the data. They 
would not go beyond the data in interpreta- 
tion and analysis. 

I wonder how often we stifle creativity and 
restrict the development of a Barbara or a 
Celia because of the ways that we make as- 
signments in our classrooms or courses, the 
kinds of things that we reward. The thing that 
I would come back to, in terms of looking at 
what both these presenters have suggested to 
us as adult educators, is that what is impor- 
tant for us is to examine our rules as instruc- 
tors or teachers to be aware ourselves of what 
we reward and to be sensitive to the kinds n f 
ways that we interact or intervene in the 
learning activities of the students with whom 
we work. 

Sharan Menriam: My comments really 
take a slightly different bend than Hue/s. 
When I thought about how I might comment 
on the two presentations, what came to mind 
was how this information intersects with what 
we have in adult education in various ways 
and how some of the information is congruent 
with some of the information we have used. It 
may be called by different names in our own 
field, but some of what we heard can also per- 
haps give us slightly new ways of looking at 
some of the same problems. 

To begin with, one thing that came out in 
both presentations that is a central credo in 
our field is the importance of the adults' prior 
experience. One speaker talked about the im- 
portance of prior experience, not just having 
it or not just the amount of the experience, 
but the ability to use it. The other speaker 
talked about the importance of prior ex- 
perience and learning from prior experience. 
We know the importance of experience in 
adult education and how we like to use adults' 
prior experience in classroom activities and in 
practice. But maybe we should be thinking 
about not just the fact that adults have more 
experience than pre-adults, but what about 
that experience? How has it been used and 
how have the adults learned from it? I think 
we can talk more about ways to extract that 



kind of information rather than just the fact 
that they have prior experience. 

Another thought that struck me this after- 
noon was that the areas which we like to 
claim as important in our field are self- 
directed learning end, of course, the Iearning- 
how-to-learn work that Bob Smith is doing at 
Northern Illinois. The aspect of self- directed 
learning that was interesting to reflect upon 
was the self-assessment that Claire talked 
about and how we say that adults are capable 
of not only directing their own learning but of 
assessing their own learning. In reality we 
don't let them do that very often. Perhaps we 
can take some of the techniques that she 
mentioned and begin to apply them more 
rigorously in terms of letting adults do their 
own self-assessment. 

In her discussion of self-assessment, 
cooperative learning was also interesting. We 
know, back from a time in the 40s and 50s 
when groups were so popular, one of the 
models was adults teaching adults and adults 
learning with adults, that cooperative learning 
is one of the mainstays of our field. I think 
that most of us here use that model extensive- 
ly in our teaching but I don't think we use it 
in terms of selx-assessment. Perhaps we could 
think of way? to integrate it into self-assess- 
ment. 

A third area that intersects with things 
going on in the field right now v/as the 
reference by both speakers to expertise versus 
novice. I know that Ron Cervero and other 
people interested in continuing professional 
education are working with the notion of ex- 
pertise and what is an expert continuing 
professional educator. What is an expert 
program planner in our field? What con- 
stitutes expert knowledge? How does one get 
to be an expert? Some works, such as Reflec- 
tive Practitioner, are tied up in the exploration 
of this 'area. 

We also have some interest, at least in our 
university, in looking at the translation of ex- 
pert knowledge such as expert computer sys- 
tems. How does one get knowledge from an 
expert and translate it into what is called one 
of the expert systems in the computer world? 
We have a couple of doctoral students doing 
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dissertations on that at the moment. 

The last point that struck me is that in- 
herent in both presentations was the notion 
that the learner is the key aspect. As a field, 
we like to think of ourselves as learner 
centered. But in both their orientations the 
learner is not just the outcome but is also a 
part of the process. Attention to the learner 
and what the learner brings to the situation- 
the types of intelligence on one hand and the 
types of experience on the other hand-is as 
important as the outcome in a learning trans- 
action. 

I would like to see more discussion from 
both of our speakers on how they see the 
adult's life situation as being different than 
your traditional college-age student. This 
strikes me as sort of aconte^tual. Those of us 
who deal with adults on a daily basis know 
that some of the factors in their learning are 
very much context-bound, life-situation deter- 
mined. I would like to see more addressing of 
those issues and how adults' learning and in- 
telligence may or may not be different from 
what I would call pre-adult or traditional col- 
lege-age students. 

Burton Sisco: I think it was Burt Kreitlow, 
in a book en itled Controversies in Adult 
Education, that described adult educators as a 
rather contentious lot. It remains to be seen if 
both Dr. Sternberg and Dr. Weinstein 
describe us that way. It seems to me that what 
they have provided is some /urther kind of 
confirmation about things such as human in- 
telligence and learning that will make us even 
more renegade-like. A number of us have 
been very uneasy about the overemphasis on 
analytic types of intelligence, or componential 
intelligence, according to one aspect of the 
triarchic theory Dr. Sternberg describes. I 
learned recently, when I sat through a 
workshop on social style, that I'm an expres- 
sive; therefore, I tend to mix facts with emo- 
tions. I suppose it's in that vein that I've been 
very uneasy with our toleration, for example, 
with the normal curve. I know there is great 
statistical veracity for that, but for me that 
just tends to acknowledge mediocrity, our 
overemphasis on testing mechanisms, and the 
idea that IQ tests are somehow free from 



bias-be they social, cultural, gender or so 
forth. 

I'd like to relate Dr. Sternberg's work to 
my own graduate students as he was describ- 
ing the three individuals, Alice, Barbara; and 
Celia. One of the connections that I have 
made is that certain graduate students seem 
to possess marvelous talent, do marvelcusly 
well in the classroom, are very punctual in 
terms of their papers, and write really good 
papers (and I don't want to define that but I 
think that most of us would tend to agree 
they're literate). Yet, as we move through the 
process from knowledge acquisition and tech- 
nical competence to the higher levels of think- 
ing where we get into synthetic analysis and 
evaluative thinking, we begin to note some 
problems. We pose questions in qualifying 
exams in a way that will enable our students 
to begin to synthesize, bring information 
together, and yet certain students have 
trouble doing this. At the sam*; time, we have 
other students who are able to bring that 
together beautifully. We then move to the dis- 
sertation stage and, again, some students 
seemingly have greater difficulty conceptualiz- 
ing a problem, working that problem out, 
figuring out a method, whereas others seem- 
ingly have no problem whatsoever. 

Personally, this is an area that I might 
suggest could be looked at in more detail. Has 
there been research that has looked at the 
phenomena of those who succeed in graduate 
study and those that don't? And, in fact, can 
we explain that in the context of the triarchic 
theory? It would seem to me that as we move 
students through the process of graduate 
study, we certainly do a good job with the 
analytic or the instrumental learning; even 
prefer it. But perhaps the practical intelligen- 
ces, the street smartness, the idea of creativity 
and novelty, the idea of becoming more self- 
directing, we do less well there. It would seem 
to me that Dr. Sternberg's triarchic theory 
may, in fact, help us to begin to account for 
this confounding situation. 

Another observation has to do with a very 
logical connection, as Sharan mentioned, with 
some of our own research in adult education. 
There have been some writings, particularly in 
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the area of self-directed learning, where 
people have tried to understand the 
phenomena in greater detail. Out of this has 
grown another kind of triarchic explanation. 
Instrumental learning is described (and th > is 
based largely on the work of Habermas) as 
where we learn to manipulate the environ- 
ment and typically involve the cognitive skills 
of knowledge and comprehension. On a 
second level, and that's the dialogic or com- 
municative learning, we come to understand 
what each other means. Thirdly, the self- 
reflective area which seems to be very ex- 
periential and includes the idea of n.U only 
acquiring experience but also utilizing thnt ex- 
perience. It seems to me that it might be 
fruitful for us to see if we can find some logi- 
cal connections between what some have said 
is unique to adulthood (communicative and 
experiential forms of learning) and Dr. 
Sternberg's triarchic theory of human intel- 
ligence. Perhaps there is also a fruitful 
relationship to Dr. Weinstein's work as well. 

I am curious; has the triarchic theory been 
tested? Is there empirical veracity to support 
its claims? I would like to leave that as a 
question at this point. Also, how, in a practi- 
cal way, can we further integrate it in the con- 
text of adulthood? 

In the context of Dr. Weinstein's emphasis 
on the learning-to-learn concept, I believe we 
must avoid sloganism and quick fixes. I feel 
somewhat relieved that she has not done that. 
It would be comforting to learn more back- 
ground of the development of the LASS! test 
and how to operationalize some factors that 
she considers to be key elements in the con- 
text of learning-to-learn and learning perfor- 
mance, especially for adults. 

It seems to me that it might be useful if 
we could also transfer the emphasis on learn- 
ing-to-learn to learning to teach. Most of us 
teach us we learn, at least that's what a lot of 
the research has suggested. The whole area 
called "learning style, or teaching style, or in- 
structional effectiveness" is very appalling in 
the context of adult education. There too 
might be some very fruitful areas in the con- 
nection between general kinds of strategies, or 
executive processes that cut across any dis- 



cipline or any presentation, and those that are 
more subject-specific. 

Finally, the notion of fluency and 
flexibility is very tricky for me. The reason is • 
that it seems to begin to account for a dimen- 
sion that describes those individuals who are 
seemingly very effective at learning in a 
general or practical way «nd less effective in 
classroom situations. It evokes a continuum 
that I think we need to study more, and that's 
the notion of versatility. What we really ought 
to be teaching or what we ought to be work- 
ing toward is certainly the rudiments of what 
might be termed learning-to-learn, but more 
so in the context of helping students become 
more versatile and flexible in their learning. 

Chcre Coggins: I took a different tact be- 
cause I'm such an affiliative learner without 
anybody to talk to, all I could do was raise 
questions, not answer questions. 

I was struck with Alice and struck with the 
fact that Alice could pass the Graduate 
Record Examination (GRE) exams and get 
wonderful scores and I sat back smugly and 
said, M I come from an institution where we 
don't use GREs and we don't use Scholastic 
Aptitude Tests (SATs)." I thought to myself, 
how can we a:, adult educators stand up and 
talk about the need for cultural diversity in 
our field yet use exams that do not allow for 
cultural diversity? How can we talk about the 
value of background and experience and the 
diversity that we as adult learners have, yet 
give a test that requires a particular set of ex- 
periences, a specific background? How can we 
talk about that fact that adults probably have 
a tendency more toward reflection than im- 
pulsivity, that they don't work quickly but they 
want to be perfectionists, that they probably 
would have skipped the last five questions 
rather than be embarrassed by guessing, yet 
we ask the 1 to take a timed test. We know 
adults have a tendency to be test anxious, yet 
we say, "Hey, listen, if you pass, if you get a 
good score of 40, we'll let you in." So I raise 
the question, how can we as adult educators 
require in good conscience the kinds of tests 
that we insist students take who enter our 
programs? 

As I thought of Barb and those people 
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who stand in judgment of her, those who had 
expectations relative to wiwit a person with a 
good GRE test score would be able to do 
relative to somebody wit!" a lesser test score, I 
started to think about the whole role of expec- 
tancy. In our research with distance learners, 
we discovered that one very telling variable is 
expectancy. Those who say, "I don't think that 
Pm going to do well," don't do *vell. What role 
does expectancy have on ths completion of 
degree programs, courses, exams? My bigger 
question relates to where does this expectancy 
come from? What pieces of information do 
people pull together to determine whether or 
not they should expect to do well or expect to 
do poorly? Expectations of others? Test 
scores? 

I have questions about Cecelia's contex- 
tual intelligence. It seemed to raise the ques- 
tion of socialization, in learning abilities, in 
reading or sensing contexts, in repertoire 
development. I was also struck with Bob's dis- 
cussion of verbal and nonverbal communica- 
tion because some of my work is in distance 
education. I started to get concerned and 
raise questions in my mind about multi- 
mediated education. How do you know I'm 
smiling when I s^nd you a message on E 
mail? How do you know I'm reall} being sar- 
castic when I send you this little cryptic note? 
How do we deal with the nonverbal com- 
munication when we deal with so many media 
as we deliver edut\ iun to adults at a dis- 
tance? 

Also, when we talked about psyching out 
professors and students working on problems 
different than we thought they were working 
on, it reminded me that as we do our experi- 
ments and have measurement issues to deal 
with, how do we know that all the subjects in 
our experiments are dealing with the samj 
problem/ How do we know they're all defin- 
ing it in the same w u y? What does k it do to 
our research results? What about the inac- 
curacy of context reading where many of uur 
stu/ients assume that we're only looking for 
surface details. They don't understand the ex- 
pectations are deep learning. I believe that 
Claire spoke to that earlier. We have to be 
veiy clear about our expectations 



Bob talked about being over-proce- 
duralized as learners. Those who become ex- 
pert learners, those who have really excelled 
in elementary and secondary school, perhaps 
are at a great disadvantage. They have ex- 
celled by following the instructor wherever he 
or she may lead. In other words, they have 
really accomplished wonders in being social- 
ized into a role that we expect them to drop 
when they come to college. At the col- 
lege/university level, we expect them to take 
on responsibility; yet, they have not been so- 
cialized to do that. They've been over-proce- 
duralized to do something else. What is 
needed to make that paradigm shift? What 
role does Outlining expectations from the out- 
set of the class have? Can we instigate 
perspective transformation? There are some 
positive things to be said about being over- 
proceduralized too. As Claire talked about the 
anxious students, those who got so anxious 
that whatever learning skills they might have 
hvj went right out the window, I wondered 
how do we over-proccduralize so that no mat- 
ter what you do to me-if I'm only banging on 
two units-those learning skills are at the same 
level as Claire's vacuuming and compulsive 
housekeeping. No matter what you do to me, 
I will still have learning skills so ingrained that 
I'll be able to rise above whatever 

Sharan focused on self-assessment; that 
was another th^me that struck me. Bob talked 
about shapers. How do we fester thoughtful 
reflection on our own study skills, our own 
learning skills, so each student can determine 
their own strengths, their own weaknesses, 
and begin to remediate or to compensate 
where necessary. 

I also wondered about knowing what you 
know and knowing what you don't know. Oft 
times we're so busy running around we never 
take time to think about that. Also, Claire had 
mentioned depicting knowledge and depicting 
relationships, and somehow that all fit 
together to me. In my free association I 
turned to thinking about different kinds of 
knowledge and branched out to concept map- 
ping. I wondered what difference it would 
make if I said to my students at the outset of 
*he class, Try to map out what you know 
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about the concepts that we're going to be dis- 
cussing in this class. Let's map out the major 
concepts and the related concepts." What dif- 
ference would it make if we had students be- 
come very conscience of what it is they know 
and what the value of that experience is which 
they bring to the classroom? What value 
would it be to ask them to go througn the 
same kind of activity half way through the 
class, to take a look at that old concept map 
and say, "Gee whiz, it looks really different. 
Now it should look like this; plus Fve added 
that and that and that"? What difference 
would it make to the nature of their learning, 
the depth of their learning, the retention of 
that learning both long term and short term? 
What difference would it make if a concept 
map was the last thir" they did in class, some- 
thing that might give them some thoughtful 
self-evaluation, an opportunity to see where 
they had come from over time? 

I was intrigued last week with peer tutors. 



Those of us who work with distance education 
students are always looking for ways to help 
people, and tutors are always one way. I think 
the sharing role of the peer tutor is wonder- 
ful; we usually talk about the one way tutor 
role. Sharan helps me; Sharan grows; I grow a 
little bit, but Sharan grows a lot. I really was 
intrigued with the back and forth situation. I 
also recalled research by Dan Coldeway up in 
Athabasca University that showed that peer 
tutors, when tutoring ten or less students, 
could accomplish as much as an expert. That's 
something to think about. 

Lastly, contextual factors are very impor- 
tant, particularly for learners who are learning 
at a distance, because their context really im- 
pinges upon them. Their classroom is the 
kitchen table in their jammies and slippers at 
two o'clock in the morning. What difference 
does that context make? Lots of questions; I 
look forward to answers. 
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Real Life vs. Academic Problem Solving 

Robert J. Sternberg 



The definition of intelligence and the 
measurement of it through typical IQ tests 
should cause some real concern for adult 
educators. There is very little similarity be- 
tween what these tests measure and the 
problems adults face in their lives. Adult 
educators who base their theories of learning 
on traditional definitions of intelligence or 
build programs on such concepts are in 
trouble, just as people who solve proolems in 
real life the way they probably were taught to 
solve problems in school get into trouble. 

What I want to talk about today is the dif- 
ference between everyday problems and 
academic or test- taking problems. And I want 
to start with the idea that in real life you have 
to learn to recognize problems. Usually 
nobody is going to come aloii^ and tell v^u 
that you have a problem--at least not until it 
is too late to do much about it. In school, a 
teacher may say, "Here is a problem to work 
on," but that is not the case in the business 
world, for example. 

If you look at the automobile industry 
back in 1974, the only foreign cars on 
American streets were Volkswagens, plus a 
few Mercedes or rich-people cars. After 1974, 
suddenly there were Hondas, Mazdas, 
Toyotas, Nissans, and so on. What happened? 
Very simply, Japanese car makers recognized 
a problem that Detroit auto makers did not 
recognize; namely, that the point would come 
when oil would be scarce and people would 
;/ant smaller, more fuel-efficient cars. Detroit, 
to the contrary, for years had been successive- 
ly upsizing cars because "the bigger the car, 
the bigger the profit." The Japanese recog- 
nized the problem; the Detroit auto makers 
did not. 

It's not enough just to recognize problems; 
you have to be able to define them. If you 
don't define them correctly, youVe got a real 
problem. My brother faced an example of 
problem definition. Some years ago, he lived 
in a house in Virginia, and right by the house 
was this big box elder tree. The aciual 



problem was thousands of disgusting, little 
brown box elder bugs all over the house. He 
was not able to tolerate the bugs, especially 
when the weather cooled off and they started 
to come inside. What do you do with this very 
real-world problem? 

This is a situation where problem defini- 
tion is key. There are a number of ways to 
define the problem, and the way you define 
the problem is going to determine the solu- 
tion. So what could you do about the box 
elder problem? You could cut the tree down. 
If you do that you're defining the problem as 
the tree. As soon as you define the problem 
as the tree, the solution immediately follows 
that you cut the tree down. But it turns out 
that there are problems with that definition of 
the problem. If you are careless, the tree 
might la* J on the house. You begin to realize 
that the tree was a nice shade tree. You also 
realize that there are other box elder trees in 
the neighborhood. If you cut one down, you 
still have the others. In this case, most impor- 
tantly, the tree was not on his property. 
Anyway, to cut down the tree does not turn 
out to be a useful definition of the problem. 

What else could you do? Calling the exter- 
minators was actually what I suggested. That 
turns out to be another way of defining the 
problem, in other words, defining it as the 
bugs. But that definition also has problems. 
The bugs come back two months later. 
Another problem is the kind of poison they 
use is fairly toxic, and my brother didn't want 
his kids zonked. The exterminator solution, 
based on defining the problem as the bugs, 
also has problems. 

A better approach is to define the 
problem as the house. One possible solution 
that follows from defining the problem as the 
house is to move. That seems to be 
problematical for the obvious reason. No one 
wants to bi» .he house given the way it looks. 
If you define it as the surface of the house 
and change the surface, in this particular case, 
that turns out to be the optimal definition. It 
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turns out that my brother noticed that all the 
houses that had box elder bugs were light 
t ored; all the houses that did not were dark. 
He painted the house brown figuring that 
either it would get rid of the bugs or he would 
not see them anymore, and, in fact, that got 
rid of the bugs. But the point to the story is 
that no one defined the problem for him. He 
had to define it himself. 

Another example is what happened to the 
United States during the Nixon administra- 
tion. Remember the Watergate fiasco where 
some burglars broke into the Democratic 
Headquarters in the Watergate Hotel and 
stole documents? Regrettably for them, they 
got caught. Nixon had a problem and the way 
he defined the problem would be absolutely 
critical to his career. In many respects he was 
intelligent and his advisors were wise in solv- 
ing problems; in defining this problem they 
were not so smart. What does that mean? The 
way they defined the problem was this terrible 
thing happened, anj it is leaking out. What 
we have to do is contain the damage by plug- 
ging leaks. If you question whether they ac- 
tually defined the problem as one of plugging 
leaks, remember there was a group of people 
working for the Committee to Re-elect the 
President who were called the "plumbers." 
Plumbers plug leaks. If you pursue the 
metaphoi further, you know that if there is 
enough water pressure, eventually the dike 
gives way. That's exactly what Nixon found. 
Even though he tried to plug the leaks, there 
was so much pressure that everyday there was 
another revelation about what was going on. 
Eventually Nixon was forced to resign, not be- 
cause of the burglary, but because of the 
coverup. In my view, it was because of the in- 
appropriate definition of the problem. 

It is another real-world example in which 
people do not define the problem for you. I 
would argue that there was a much better 
definition of the problem than the one Nixon 
used. The better definition was to admit what 
happened and get it all out. People accused 
Nixon of being Machiavellian, but in The 
Prince Machiavelli said: "If you have bad 
news, tell the people all at once. If you have 
good news, dribble it out slowly." Nixon had 



bad news and he dribbled it out slowly. Some 
might say if he admitted it all at once he still 
would have had to resign. But I want to argue 
that there are several recent examples where 
people did use the Machiavellian principle, 
and it worked out fine. 

A few months ago it was discovered that 
Chrysler executives were using new Chrysler 
cars for their own personal use, and after iliey 
used them the odometers were turned back 
and they were sold as new ca^. In some cases 
the cars had been wrecked. They were 
repaired and still sold as new cars. Unfor- 
tunately for Chrysler, it gets out. So what do 
you do about this embarrassing incident? One 
possibility is to try to control the leaks. Iacoc- 
ca chose the opposite. He took out full-page 
ads in major U.S. newspapers, admitting that 
they had goofed, that they were embarrassed, 
and that it would not happen again. After a 
few weeks, no one was talking about Chrysler 
rolling back odometers on cars. 

A more serious example was the Tylenol 
fiasco. You may remember that a few years 
ago somebody decided to lace extra-strength 
Tylenol with a permanent cure to headaches. 
As a result, a lot of people did not want to 
buy Tylenol anymore because they weren't 
looking for anything that long-lasting. The 
Tylenol company, Johnson and Johnson, now 
had a problem. They could have tried to cover 
it up. Instead they chose to admit the problem 
and recall all Tylenol. People said they were 
crazy; Tylenol was the leading se *ng anal- 
gesic in the United States, and no one would 
ever buy Tylenol again. What is the leading 
selling analgesic in the United States today? 
Tylenol. 

^ j, again, the definition of the problem is 
more critical in many ways than the way it is 
solved. But, if children are taught that they 
are always going to have problems defined for 
them, as is done often on tests or in school, 
they are not going to be ready for practical 
problems. 

What I have been arguing is that two of 
the things that schools are doing, pre-recog- 
nizing and pre-defining problems, do not work 
outside the school setting. A third difference 
between everyday problems and v/hat is being 
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taught in schools and in testing situations is 
the issue of structureless of problems. I 
want to introduce this by telling you a story 
that happened to me as an adolescent. 

A newspaper had an advertisement for 
handwriting analysis. I had fairly unusual 
handwriting, so I decided to send away for a 
$10.98 handwriting analysis. A couple of 
weeks later I got back the handwriting 
analysis. Some of you probably believe that 
there is nothing to handwriting analysis, but I 
wasn't sure what I believed. To my 
astonishment, the analysis was really good. It 
wasn't like the crystal ball gazers where they 
say things about you that make you feel good 
but which could have been said about anyone. 
Rather, the analysis said things of me that 
would not be true of other people, things that 
the person could not possibly have known ex- 
cept by analyzing the handwriting. They said 
things like I have unusual leadership abilities 
though Pm not always given an opportunity to 
exercise them; I'm creative although people 
don't always recognize this; Pm compas- 
sionate and caring of other people! 

The questions this raises is, how they 
could make a profit doing this for $10.98 per 
throw? It turns out there was a little message 
in with the handwriting analysis which said: 
You will notice that you have a favorable per- 
sonality structure, but that there are a few 
things that you may want to improve about 
your personality. We will help yoc* do that if 
you send $69.95. We will give you handwriting 
lessons and, by changing your handwriting, 
give you a bettei personality. 

The logic in this example is similar to the 
logic we use for increasing children's or 
adults' intellectual skills. What wc do to make 
people smarter is basically what is recom- 
mended in the handwriting story. That sounds 
sort of odd so let me say why I believe that 
that argument is correct. When people first 
theorized abo *t and created intelligence tests, 
they did not believe that what you find on in- 
telligence tests it; the stuff of intelligence. To 
the contrary, Benet and Wechsler said that in- 
telligence is the ability to adapt to the 
everyday situations. The tests are an index of 
that ability. It's like a thermometer; no one 
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really believes the thermometer has the heat 
itself; it is only an indicator of the heat. A 
fever is not a disease; it is just a measure of 
that disease. The analogy is if you give the 
person aspirin, you m* y reduce the fever, the 
symptom, but you would not be so foolish as 
to say that you are getting rid of the disease. 
You don't cure a virus by giving a person 
aspirin; you just lower the fever. 

Similarly, increasing scores on intelligence 
tests does not increase intelligence. You 
change the assessment, the index measure, 
not the intelligence itself. Obviously it's not 
bad to increase scores on intelligence tests. If 
you want to get into college or graduate 
school you need high scores on tests. It would 
be an innocuous business if that's all there 
was to it, but in some cases, what is required 
on the intelligence tests may actually go 
against real-world intelligence. Then what you 
may be doing is teaching people skills that do 
not work outside of testing and some 
academic situations. Recognition of problems 
and definition of problems are two examples. 
The third is structuring of the problems. 

IQ tests, achievement tests, and academic 
problems tend to be well-structured. There is 
a path, and if you follow that path, you're 
guaranteed a solution. There has to be, or 
people would complain to ETS or Psych Corp 
that it is ambiguous. Teachers worry about 
giving problems that are too ambiguous; how 
do you score them? Despite the fact that test 
problems and most academic problems are 
well-defined, very few real-world problems are 
defined. For examp } * tne problem of improv- 
ing your personality is not ~ 1 d by changing 
your handwriting. 

The best selling book you can find is 
called Ten Easy Steps to..." In fact, of all the 
books I've written, the most successful has 
been How to Improve Your Score on the Miller 
Analogies Test. Why? Because it listed ten 
easy steps on how to improve your scores on 
the Miller Analogies Test. Look at the best 
seller list; you'll see books on how to lose 
weight, how to stop smoking, and so forth. 
The fact is that they do work. You do lose the 
weight for a week and then you gain it back. 
You stop smoking for a week and then you 
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start ag^in. Things do not work ^ any long- 
term basis. There is a very simple test that ab- 
solutely proves I'm right. The books keep sell- 
ing and new books keep appearing. But if any 
of it worked, you wouldn't need new books. 
There would be one method, and people 
could use it to lose the weight or stop smok- 
ing once and for all. 

Real-vorld problems are ill-structured, not 
well-stiuctured. Teaching children or adults to 
be smart by solving well-structured problems 
is a disservice because that's not the way life 
is. It teaches them to look for the "ten easy 
steps solution" and then to faii again and 
again. Another difference between test and 
academic problems and what you find outside 
of a school setting is contextualization. 
Problems on tests are decontextualized. 

Similarly, problems in textbooks are all 
unrelated to each other. There is very little in- 
formation in any of the problems. You're sup- 
posed to solve the* problem on the basis of 
three or four sentences, sometimes less. Real- 
world problems are not decontextualized; 
they're contextualized. You can't solve them 
in isolation from other variables. If you're 
thinking about buying a house you can't solve 
it in three or four sentences. You have to take 
into account interest rates, house prices, what 
your spouse wants, your job, how much money 
you have. There are a lot of contextual factors 
that affect the decision. 

Problems of any consequence in everyday 
life differ greatly from problems you find in 
school and on tests. You could say "Big deal!" 
if people solve decontextualized problems the 
same that they solve contextualized problems. 
There is much evidence that it is not the same 
at all. For example, psychologists built a 
theory of decision making on the basis of ex- 
periments done with poker chips. They had 
people choose poker chips; if you chose the 
right one you got three cents; if you chose the 
wrong one you lost two cents. By the end of 
the experiment a person might gain or lose up 
to $2.00. From this they built a theory of 
decision making. Then other psychologists 
asked people to make real-world decisions, 
and they discovered something very extraordi- 
nary. The whole theory of decision making 



built on poker chips worked only for poker 
chips. 

When I worked for the Psychological Cor- 
poration we had a complaint about a college 
that was using the Miller Analogies Test for 
admissions. The school required a score of 25 
for admission, which could have been the 
basis for a complaint because it is a 100-item 
test with four multiple choice options which 
makes the chance score 25. But the actual 
complaint was by a woman who was admitted 
with a score of 24 because her other creden- 
tials were so exceptional. Now that she has 
finished the program and is ready to graduate 
with honors, they do not want to give her the 
degree. Why? She did not get a score of 25 on 
the Miller Analogies Test. They asked her t d 
retake the Miller Analogies Test. The predic- 
tor again becomes more important than the 
criteria to be predicted. This case had a happy 
outcome. The woman retook the Miller Anal- 
ogy Test, got a 26, and was given her diploma. 

This kind of thing is typical of talk about 
overachievers. I've heard it said that some 
person is an overachiever, not really very 
smart, but for some reason more successful 
than deserved. The whole concept is 
ridiculous because it is based on a very nar- 
row measurement of intelligence. If someone 
does better than predicted by this narro>v test, 
then; must be something wrong with them. 
You do not have to fix the test; you have to 
fix them so that their performance is more in 
line with what is shown by the IQ test! 

There is a fourth difference in the way 
people solve contextualized problems. I or- 
ganized a study in which college students were 
given IQ test-like problems and also real- 
world problems. An example of a real-world 
problem is the following. Josh and Sandy were 
discussing two baseball teams, the Reds and 
the Blues. Josh laid every member of the Red 
team is better than every member of the Blue 
team which proves the Reds are the better 
team. Is this conclusion valid? This kind of 
reasoning is called the fallacy of composition. 
The fallacy rests on the belief that the whole 
equals the sum of the parts. But any number 
of instances can be found where that does not 
work. George Steinbrenner bought the New 
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York Yankees some years ago and populated 
his learn with the very best players for each 
position. The next few years they won few 
games. The players were good, but they could 
not work together. 

A fifth difference between academic 
problems and real-world problems is that 
most school problems have one right answer. 
All test problems have one right answer. Very 
few real-life problems have one right answer. 
You cannot say, the right way to solve the Is- 
raeli/Arab conflict is...; or the right way to 
deal with an angry spouse is.... There are op- 
tions that work better or worse, but there is 
certainly no right answer. Teaching children 
that problems have right answers may work 
for test problems, but not for many real-life 
problems, 

A sixth difference between academic 
problems and everyday ones is that of 
relevant and available information. On a test, 
the information that will solve the problem 
must be found in the test problem or 
presumed to be known by the test takers. It 
would be nice to have it that way in life. Did 
you ever try to find a really good doctor--a 
surgeon or r specialist? How do you recog- 
nize a good doctor? Wher* do you get the in- 
formation that would tell you who is a good 
doctor? Do you look at the medical school at- 
tended or the number of survivors of opera- 
tions performed? Do you call the American 
Medical Association for a list of members of 
the Medical Association? The point is that in 
the real world it is not clear where to get the 
information or even what exactly the informa- 
tion is that you need. 

The seventh difference is most easily il- 
lustrated by examining a very typical 10 test 
problem. The probleir is sometimes called the 
2-4-6 problem and it is on virtually every in- 
telligence test. What the test taker must do is 
extrapolate the series. We'll take a simple 
series to illustrate the point. You are given 2- 
4-6 and you have to say what number comes 
next. The idea is that there is a series of 2-4- 
6-8-10-12-14 and a rule based on those num- 
bers. The rule is increasing even numbers or 
numbers increasing by twos. If you write that 
answer on the test, you get it right. 



However, increasing even numbers by two 
is not the rule that I have in mind. So that 
raises a problem; what is the rule I have in 
mind? Give me some more numbers. 

•Twelve." Right, that's in the series. 

"Fourteen." In the series. 

"Thirty-six." In the series. 

"Thirteen." Also in the series! Stop for a 
minute and think about the difference be- 
tween 13 and 8-10-12-14-16-18. The difference 
is very clear. However, every number that you 
have here, except 13, tends to confirm what 
you already believed, that is that the series is 
increasing in number by two. But then I told 
you that was not right. Yet the first responses 
I got were more attempts to confirm what you 
already knew was not true. It usually takes 
people awhile to disconfirm a hypothesis 
which they now know is wrong. It took about 
10 responses until someone suggested a num- 
ber, such as 13, 7, or 11 that disconfirms what 
was believed in the first place. Does anyone 
want to guess what the rule is? Seven is good; 
9 is good; 3 is not good; 11 is okay. The rule 
is simply increasing natural numbers. These 
three numbers 2-4-6, are three not particularly 
representative numbers from the population 
of increasing natur ! numbers. It happens that 
they are three even numbers; I could have 
used 2-4-5, 2-4-8, 2-4-10, 2-4-11. The impor- 
tant thought is that small samples are 
notoriously biased. Anyone who has had even 
one statistics course knows that. This sample 
happens to be biased toward even numbers. 

Now, I am not really interested in that 
number series. What I am interested in is that 
what we learn in school is to accept a belief 
that we are given or to accept a fact on 
authority. When we're taught to think, we are 
taught to justify what we believe. What I want 
to argue is that that is inadequate. What hap- 
pens as a result of such thinking is if you go 
to a Republican rally you become a 
Republican, but if you go to a Democratic 
rally, you become a Democrat. People seek 
out information to confirm what they believed 
anyway. They rarely seek out information that 
disconfirms beliefs, and we scientists are no 
better than anyone else. We do experiments 
to confirm what we already believe. How 
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often did I try to disconfirm my triarchic 
theory? The result is that we are not very 
good at disaffirming our beliefs and thus we 
have trouble not only understanding other 
people's views but even admitting another 
view is at all plausible. In the real world, we 
need disconfirmation as well as confirmation. 

When I taught my thinking skills course 
last ye^r to teachers from around the country, 
I gave them an exercise in which they were to 
take some belief they had, a belief that they 
were serious about and thought they could 
justify, and write it down. They, of course, 
thought the assignment was going to be to jus- 
tify the belief. No, it was to argue against 
their belief in a convincing way. The interest- 
ing thing to me was that most of those well- 
educated teachers admitted having never 
seriously considered arguments for the other 
side. Again, there is a difference between real 
world and academic problem solving. 

An eighth difference between the 
academic problem solving and everyday 
problem solving lies in feedback. In school 
settings you get clear feedback about what is 
right and what is wrong. You get Cs and X's 
and checks; you get grades on papers; you get 
feedback that is quite explicit. Unfortunately, 
that's not what happens in everyday Iili*. It is 
rare to get explicit feedback on how you are 
doing. For long periods of time on the job, 
feedback is not clear. When you're in a 
relationship with someone, it is rather rare to 
be given feedback directly from the person as 
to hew the relationship is going. Usually it is 
not until things are pretty bad that the other 
person gives you the feedback. You know, 
Tm leaving tomorrow," or Til pay you to 
leave tomorrow." Feedback in everyday situa- 
tions is very muddled, if it is given at all. 
What we must learn is how to operate with 
incomplete or unclear feedback. 

A ninth difference between the everyday 
situation and the academic situation is the 
emphasis given individual problem solving. 
The whole academic structure is geared 
toward problem solving by J he individual. If 
you work on a test with someone else, it is 
cheating. ETS actually has uniformed security 
guards who go after people who collaborate. 



So what we learn is how to do individual 
problem solving. This is disturbing because 
very little real world, adult problem solving is 
individual. Almost always people either work 
with others to solve problems or get their 
solutions cleared by other people. If you can't 
convince superiors or committees that your 
solution is a viable one, no matter how good 
it is, that is the end of it. In everyday life, very 
good solutions go down the drain because 
people cannot convince others that the solu- 
tions are good. 

Individual problem-solving skills do not 
transfer well to group settings. In fact, some- 
times you get negative transfer. An example 
for me was the first faculty meeting I went to 
as an assistant professor at Yale. All these 
faculty members were outstanding individual 
problem solvers or they would not have been 
hired. But the faculty meeting was like a cir- 
cus. There were people who were grandstand- 
ing; nobody was listening; most were busy 
preparing their own arguments. It was clear 
that these people did not know how to work 
together. There have been many other set- 
tings that I have been in since then where the 
same thing happened. In politics you get in- 
credibly poor decisions by smart people. How 
can people who are so smart, who are well 
trained, come up with stupid solutions? The 
answer is something called "group think." 
Smart people get together in a group and suc- 
cumb to group decision making. An example 
of "group think" is the formation of "mind 
guards." Someone in a group assumes the job 
to ostracize or at least criticize people who do 
not go along with the group. It happens in 
every kind of organization all the time. If you 
do not go with the group, slowly you are 
edged out of it, and eventually you lose 
whatever power you once had. 

So, I think the meaningful challenge to 
adult educators is to help adults try to solve 
their real-life problems. The kind of problems 
you find in test-taking or academic situations 
are usually not real problems and certainly 
are not presented in the same manner in 
which they occur in everyday life. I hope the 
ideas I shared with you today help you do 
that. 
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